|
»
»
Archiva 1.2.1 - repos won't get index
|
View:
New views
10 Messages
—
Rating Filter:
Alert me
|
 |
Archiva 1.2.1 - repos won't get index
by Marc Lustig
::
Rate this Message:
We upgraded our test-server to release 1.2.1, removed .index and .indexer from all the repos, ran the Scan for the repos and then the Update Database.
As a result, before the upgrade we had 500 MB inside in all of the .index directories. After the upgrade and the repo-scan, the total size is less than 1 MB. Apparently the repo indexing-process does not work anymore. The Search-function does not give any results. Any idea how to make the repos to get indexed again? |
|
|
Re: Archiva 1.2.1 - repos won't get index
by Marc Lustig
::
Rate this Message:
I forgot to add an excerpt from the log:
2009-06-03 17:42:11,554 [pool-2-thread-1] INFO org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor - Finished repository task: .\ Scan of internal \.__________________________________________ Repository Dir : /opt/managed_repos/internal Repository Name : .... Repository Layout : default Known Consumers : (7 configured) auto-rename metadata-updater repository-purge auto-remove update-db-artifact create-missing-checksums index-content Invalid Consumers : <none> Duration : 1 Minute 13 Seconds 689 Milliseconds When Gathered : 6/3/09 5:42 PM Total File Count : 107263 Avg Time Per File : ______________________________________________________________ 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor - Executing task from queue with job name: database-job:user-requested 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor - Task: Updating unprocessed artifacts 2009-06-03 17:42:55,318 [http-8080-6] INFO org.apache.maven.archiva.web.action.admin.SchedulerAction - [ActionMessage] Your request to update the database has been queued. 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor - Task: Updating processed artifacts 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor - Finished database task in 85035ms.
|
|
|
Re: Archiva 1.2.1 - repos won't get index
by Deng Ching-2
::
Rate this Message:
Hi Marc,
Did you touch the contents of the repository? The last modified date of the artifacts need to be more recent than the last run of the repository scan. You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch *at the base dir where your repo is located. Thanks, Deng On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig <ml@...> wrote: > > I forgot to add an excerpt from the log: > > 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO > > org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor > - Finished repository task: > .\ Scan of internal \.__________________________________________ > Repository Dir : /opt/managed_repos/internal > Repository Name : .... > Repository Layout : default > Known Consumers : (7 configured) > auto-rename > metadata-updater > repository-purge > auto-remove > update-db-artifact > create-missing-checksums > index-content > Invalid Consumers : <none> > Duration : 1 Minute 13 Seconds 689 Milliseconds > When Gathered : 6/3/09 5:42 PM > Total File Count : 107263 > Avg Time Per File : > ______________________________________________________________ > 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO > > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > - Executing task from queue with job name: database-job:user-requested > 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO > > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > - Task: Updating unprocessed artifacts > 2009-06-03 17:42:55,318 [http-8080-6] INFO > org.apache.maven.archiva.web.action.admin.SchedulerAction - > [ActionMessage] > Your request to update the database has been queued. > 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO > > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > - Task: Updating processed artifacts > 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO > > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > - Finished database task in 85035ms. > > > > Marc Lustig wrote: > > > > We upgraded our test-server to release 1.2.1, removed .index and .indexer > > from all the repos, ran the Scan for the repos and then the Update > > Database. > > > > As a result, before the upgrade we had 500 MB inside in all of the .index > > directories. > > After the upgrade and the repo-scan, the total size is less than 1 MB. > > Apparently the repo indexing-process does not work anymore. The > > Search-function does not give any results. > > > > Any idea how to make the repos to get indexed again? > > > > -- > View this message in context: > http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html > Sent from the archiva-users mailing list archive at Nabble.com. > > |
|
|
Re: Archiva 1.2.1 - repos won't get index
by Wendy Smoak-3
::
Rate this Message:
... and if you dislike losing the info about when a file was actually
deployed or changed, consider voting for (or patching!) MRM-1056 or the related issue: http://jira.codehaus.org/browse/MRM-1056 -- Wendy On Wed, Jun 3, 2009 at 5:50 PM, Deng Ching <oching@...> wrote: > Hi Marc, > > Did you touch the contents of the repository? The last modified date of the > artifacts need to be more recent than the last run of the repository scan. > You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch *at the base > dir where your repo is located. > > Thanks, > Deng |
|
|
Re: Archiva 1.2.1 - repos won't get index
by Marc Lustig
::
Rate this Message:
Indeed,
find [REPOSITORY_DIRECTORY_NAME] | xargs touch is tremendously faster than find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \;
|
|
|
Re: Archiva 1.2.1 - repos won't get index
by James William Dumay
::
Rate this Message:
I have the same problem with our installation. We followed the upgrade
notes and the search index never gets populated. James Marc Lustig wrote: > Indeed, > find [REPOSITORY_DIRECTORY_NAME] | xargs touch > is tremendously faster than > find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \; > > > > > Deng Ching-2 wrote: > >> Hi Marc, >> >> Did you touch the contents of the repository? The last modified date of >> the >> artifacts need to be more recent than the last run of the repository scan. >> You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch *at the >> base >> dir where your repo is located. >> >> Thanks, >> Deng >> >> On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig<ml@...> wrote: >> >> >>> I forgot to add an excerpt from the log: >>> >>> 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO >>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor >>> - Finished repository task: >>> .\ Scan of internal \.__________________________________________ >>> Repository Dir : /opt/managed_repos/internal >>> Repository Name : .... >>> Repository Layout : default >>> Known Consumers : (7 configured) >>> auto-rename >>> metadata-updater >>> repository-purge >>> auto-remove >>> update-db-artifact >>> create-missing-checksums >>> index-content >>> Invalid Consumers :<none> >>> Duration : 1 Minute 13 Seconds 689 Milliseconds >>> When Gathered : 6/3/09 5:42 PM >>> Total File Count : 107263 >>> Avg Time Per File : >>> ______________________________________________________________ >>> 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO >>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>> - Executing task from queue with job name: database-job:user-requested >>> 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO >>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>> - Task: Updating unprocessed artifacts >>> 2009-06-03 17:42:55,318 [http-8080-6] INFO >>> org.apache.maven.archiva.web.action.admin.SchedulerAction - >>> [ActionMessage] >>> Your request to update the database has been queued. >>> 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO >>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>> - Task: Updating processed artifacts >>> 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO >>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>> - Finished database task in 85035ms. >>> >>> >>> >>> Marc Lustig wrote: >>> >>>> We upgraded our test-server to release 1.2.1, removed .index and >>>> >>> .indexer >>> >>>> from all the repos, ran the Scan for the repos and then the Update >>>> Database. >>>> >>>> As a result, before the upgrade we had 500 MB inside in all of the >>>> >>> .index >>> >>>> directories. >>>> After the upgrade and the repo-scan, the total size is less than 1 MB. >>>> Apparently the repo indexing-process does not work anymore. The >>>> Search-function does not give any results. >>>> >>>> Any idea how to make the repos to get indexed again? >>>> >>>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html >>> Sent from the archiva-users mailing list archive at Nabble.com. >>> >>> >>> >> > > |
|
|
Re: Archiva 1.2.1 - repos won't get index
by Marc Lustig
::
Rate this Message:
On our side, the story is continuing like this.
In the afternoon, I touch the whole repo and let the scan running and re-create the database. After that, it seems like all (physically) existing artifacts can be found on the Web-UI. Fine. The next morning again hundreds of artifacts are disappeared from the Web-UI (but still existing physically). It's so annoying... I checked the last couple of scans in the log and cannot find exceptions for that Repo concerned. (However I didn't check all the scans since yesterday afternoon.) Anyway, for some reason, some consumer is removing certain artifacts (or whole groupId's) from the index. I have no clue why. Any suggestion what to do to prevent the removal of artifacts from the index-database? If we disable the "update-db-artifact" consumer, what undesired side-effects may this have?
|
|
|
Re: Archiva 1.2.1 - repos won't get index
by Deng Ching-2
::
Rate this Message:
On Thu, Jun 25, 2009 at 7:26 PM, Marc Lustig <ml@...> wrote:
> > On our side, the story is continuing like this. > In the afternoon, I touch the whole repo and let the scan running and > re-create the database. > After that, it seems like all (physically) existing artifacts can be found > on the Web-UI. Fine. > The next morning again hundreds of artifacts are disappeared from the > Web-UI > (but still existing physically). It's so annoying... > I checked the last couple of scans in the log and cannot find exceptions > for > that Repo concerned. > (However I didn't check all the scans since yesterday afternoon.) > > Anyway, for some reason, some consumer is removing certain artifacts (or > whole groupId's) from the index. I have no clue why. > > Any suggestion what to do to prevent the removal of artifacts from the > index-database? > > If we disable the "update-db-artifact" consumer, what undesired > side-effects > may this have? > > basic artifact info in the db and that's where the artifacts shown in the Browse page is gathered.. You can try disabling these consumers in Database page as they are the ones doing the cleanup work, but afaik they only clean up the db and index for artifacts that are no longer in the filesystem: not-present-remove-db-artifact not-present-remove-db-project not-present-remove-indexed If the problem still persists, do you mind filing an issue for this in jira? :) Thanks, Deng > > Marc Lustig wrote: > > > > Indeed, > > find [REPOSITORY_DIRECTORY_NAME] | xargs touch > > is tremendously faster than > > find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \; > > > > > > > > > > Deng Ching-2 wrote: > >> > >> Hi Marc, > >> > >> Did you touch the contents of the repository? The last modified date of > >> the > >> artifacts need to be more recent than the last run of the repository > >> scan. > >> You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch *at the > >> base > >> dir where your repo is located. > >> > >> Thanks, > >> Deng > >> > >> On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig <ml@...> wrote: > >> > >>> > >>> I forgot to add an excerpt from the log: > >>> > >>> 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO > >>> > >>> > org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor > >>> - Finished repository task: > >>> .\ Scan of internal \.__________________________________________ > >>> Repository Dir : /opt/managed_repos/internal > >>> Repository Name : .... > >>> Repository Layout : default > >>> Known Consumers : (7 configured) > >>> auto-rename > >>> metadata-updater > >>> repository-purge > >>> auto-remove > >>> update-db-artifact > >>> create-missing-checksums > >>> index-content > >>> Invalid Consumers : <none> > >>> Duration : 1 Minute 13 Seconds 689 Milliseconds > >>> When Gathered : 6/3/09 5:42 PM > >>> Total File Count : 107263 > >>> Avg Time Per File : > >>> ______________________________________________________________ > >>> 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO > >>> > >>> > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > >>> - Executing task from queue with job name: database-job:user-requested > >>> 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO > >>> > >>> > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > >>> - Task: Updating unprocessed artifacts > >>> 2009-06-03 17:42:55,318 [http-8080-6] INFO > >>> org.apache.maven.archiva.web.action.admin.SchedulerAction - > >>> [ActionMessage] > >>> Your request to update the database has been queued. > >>> 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO > >>> > >>> > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > >>> - Task: Updating processed artifacts > >>> 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO > >>> > >>> > org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor > >>> - Finished database task in 85035ms. > >>> > >>> > >>> > >>> Marc Lustig wrote: > >>> > > >>> > We upgraded our test-server to release 1.2.1, removed .index and > >>> .indexer > >>> > from all the repos, ran the Scan for the repos and then the Update > >>> > Database. > >>> > > >>> > As a result, before the upgrade we had 500 MB inside in all of the > >>> .index > >>> > directories. > >>> > After the upgrade and the repo-scan, the total size is less than 1 > >>> MB. > >>> > Apparently the repo indexing-process does not work anymore. The > >>> > Search-function does not give any results. > >>> > > >>> > Any idea how to make the repos to get indexed again? > >>> > > >>> > >>> -- > >>> View this message in context: > >>> > http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html > >>> Sent from the archiva-users mailing list archive at Nabble.com. > >>> > >>> > >> > >> > > > > > > -- > View this message in context: > http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p24201482.html > Sent from the archiva-users mailing list archive at Nabble.com. > > |
|
|
Re: Archiva 1.2.1 - repos won't get index
by Marc Lustig
::
Rate this Message:
Hi,
I gave this another try by doing the following steps. 1. deleting repo/.index and repo/.indexer 2. touching all files and dirs in repo recursively 3. running the repo scan When we search now for e.g. xmlbeans, we do get a few jar-artifacts that contain a package-name "xmlbeans". But we do not get the artifact whose name is xmlbeans. I have no idea what else to try to make the search-function to work again. User's here are pretty annoyed about it. Any suggestion what to do except filing a jira issue?
|
|
|
Re: Archiva 1.2.1 - repos won't get index
by brettporter
::
Rate this Message:
Perhaps you could use Luke to inspect the Lucene index and see whether
that helps identify the issue by looking at what data has been successfully recorded? On 17/09/2009, at 1:19 AM, Marc Lustig wrote: > > Hi, > I gave this another try by doing the following steps. > > 1. deleting repo/.index and repo/.indexer > 2. touching all files and dirs in repo recursively > 3. running the repo scan > > When we search now for e.g. xmlbeans, we do get a few jar-artifacts > that > contain a package-name "xmlbeans". But we do not get the artifact > whose name > is xmlbeans. > > I have no idea what else to try to make the search-function to work > again. > User's here are pretty annoyed about it. > > Any suggestion what to do except filing a jira issue? > > > > > > Deng Ching-2 wrote: >> >> On Thu, Jun 25, 2009 at 7:26 PM, Marc Lustig <ml@...> >> wrote: >> >>> >>> On our side, the story is continuing like this. >>> In the afternoon, I touch the whole repo and let the scan running >>> and >>> re-create the database. >>> After that, it seems like all (physically) existing artifacts can be >>> found >>> on the Web-UI. Fine. >>> The next morning again hundreds of artifacts are disappeared from >>> the >>> Web-UI >>> (but still existing physically). It's so annoying... >>> I checked the last couple of scans in the log and cannot find >>> exceptions >>> for >>> that Repo concerned. >>> (However I didn't check all the scans since yesterday afternoon.) >> >> >>> >>> Anyway, for some reason, some consumer is removing certain >>> artifacts (or >>> whole groupId's) from the index. I have no clue why. >>> >>> Any suggestion what to do to prevent the removal of artifacts from >>> the >>> index-database? >>> >>> If we disable the "update-db-artifact" consumer, what undesired >>> side-effects >>> may this have? >>> >>> >> Your artifacts won't show up in the webapp browse as this consumer >> adds >> the >> basic artifact info in the db and that's where the artifacts shown >> in the >> Browse page is gathered.. >> >> You can try disabling these consumers in Database page as they are >> the >> ones >> doing the cleanup work, but afaik they only clean up the db and >> index for >> artifacts that are no longer in the filesystem: >> not-present-remove-db-artifact >> not-present-remove-db-project >> not-present-remove-indexed >> >> If the problem still persists, do you mind filing an issue for this >> in >> jira? >> :) >> >> Thanks, >> Deng >> >> >>> >>> Marc Lustig wrote: >>>> >>>> Indeed, >>>> find [REPOSITORY_DIRECTORY_NAME] | xargs touch >>>> is tremendously faster than >>>> find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \; >>>> >>>> >>>> >>>> >>>> Deng Ching-2 wrote: >>>>> >>>>> Hi Marc, >>>>> >>>>> Did you touch the contents of the repository? The last modified >>>>> date >>> of >>>>> the >>>>> artifacts need to be more recent than the last run of the >>>>> repository >>>>> scan. >>>>> You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch >>>>> *at >>> the >>>>> base >>>>> dir where your repo is located. >>>>> >>>>> Thanks, >>>>> Deng >>>>> >>>>> On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig <ml@...> >>> wrote: >>>>> >>>>>> >>>>>> I forgot to add an excerpt from the log: >>>>>> >>>>>> 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO >>>>>> >>>>>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor >>>>>> - Finished repository task: >>>>>> .\ Scan of internal \.__________________________________________ >>>>>> Repository Dir : /opt/managed_repos/internal >>>>>> Repository Name : .... >>>>>> Repository Layout : default >>>>>> Known Consumers : (7 configured) >>>>>> auto-rename >>>>>> metadata-updater >>>>>> repository-purge >>>>>> auto-remove >>>>>> update-db-artifact >>>>>> create-missing-checksums >>>>>> index-content >>>>>> Invalid Consumers : <none> >>>>>> Duration : 1 Minute 13 Seconds 689 Milliseconds >>>>>> When Gathered : 6/3/09 5:42 PM >>>>>> Total File Count : 107263 >>>>>> Avg Time Per File : >>>>>> ______________________________________________________________ >>>>>> 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO >>>>>> >>>>>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>>>>> - Executing task from queue with job name: >>> database-job:user-requested >>>>>> 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO >>>>>> >>>>>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>>>>> - Task: Updating unprocessed artifacts >>>>>> 2009-06-03 17:42:55,318 [http-8080-6] INFO >>>>>> org.apache.maven.archiva.web.action.admin.SchedulerAction - >>>>>> [ActionMessage] >>>>>> Your request to update the database has been queued. >>>>>> 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO >>>>>> >>>>>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>>>>> - Task: Updating processed artifacts >>>>>> 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO >>>>>> >>>>>> >>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor >>>>>> - Finished database task in 85035ms. >>>>>> >>>>>> >>>>>> >>>>>> Marc Lustig wrote: >>>>>>> >>>>>>> We upgraded our test-server to release 1.2.1, removed .index and >>>>>> .indexer >>>>>>> from all the repos, ran the Scan for the repos and then the >>>>>>> Update >>>>>>> Database. >>>>>>> >>>>>>> As a result, before the upgrade we had 500 MB inside in all of >>>>>>> the >>>>>> .index >>>>>>> directories. >>>>>>> After the upgrade and the repo-scan, the total size is less >>>>>>> than 1 >>>>>> MB. >>>>>>> Apparently the repo indexing-process does not work anymore. The >>>>>>> Search-function does not give any results. >>>>>>> >>>>>>> Any idea how to make the repos to get indexed again? >>>>>>> >>>>>> >>>>>> -- >>>>>> View this message in context: >>>>>> >>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html >>>>>> Sent from the archiva-users mailing list archive at Nabble.com. >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p24201482.html >>> Sent from the archiva-users mailing list archive at Nabble.com. >>> >>> >> >> > > -- > View this message in context: http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p25474649.html > Sent from the archiva-users mailing list archive at Nabble.com. > |
| Free embeddable forum powered by Nabble | Forum Help |
