|
»
« Return to Thread: Archiva 1.2.1 - repos won't get index
Re: Archiva 1.2.1 - repos won't get index
by brettporter
::
Rate this Message:
Perhaps you could use Luke to inspect the Lucene index and see whether
that helps identify the issue by looking at what data has been
successfully recorded?
On 17/09/2009, at 1:19 AM, Marc Lustig wrote:
>
> Hi,
> I gave this another try by doing the following steps.
>
> 1. deleting repo/.index and repo/.indexer
> 2. touching all files and dirs in repo recursively
> 3. running the repo scan
>
> When we search now for e.g. xmlbeans, we do get a few jar-artifacts
> that
> contain a package-name "xmlbeans". But we do not get the artifact
> whose name
> is xmlbeans.
>
> I have no idea what else to try to make the search-function to work
> again.
> User's here are pretty annoyed about it.
>
> Any suggestion what to do except filing a jira issue?
>
>
>
>
>
> Deng Ching-2 wrote:
>>
>> On Thu, Jun 25, 2009 at 7:26 PM, Marc Lustig <ml@...>
>> wrote:
>>
>>>
>>> On our side, the story is continuing like this.
>>> In the afternoon, I touch the whole repo and let the scan running
>>> and
>>> re-create the database.
>>> After that, it seems like all (physically) existing artifacts can be
>>> found
>>> on the Web-UI. Fine.
>>> The next morning again hundreds of artifacts are disappeared from
>>> the
>>> Web-UI
>>> (but still existing physically). It's so annoying...
>>> I checked the last couple of scans in the log and cannot find
>>> exceptions
>>> for
>>> that Repo concerned.
>>> (However I didn't check all the scans since yesterday afternoon.)
>>
>>
>>>
>>> Anyway, for some reason, some consumer is removing certain
>>> artifacts (or
>>> whole groupId's) from the index. I have no clue why.
>>>
>>> Any suggestion what to do to prevent the removal of artifacts from
>>> the
>>> index-database?
>>>
>>> If we disable the "update-db-artifact" consumer, what undesired
>>> side-effects
>>> may this have?
>>>
>>>
>> Your artifacts won't show up in the webapp browse as this consumer
>> adds
>> the
>> basic artifact info in the db and that's where the artifacts shown
>> in the
>> Browse page is gathered..
>>
>> You can try disabling these consumers in Database page as they are
>> the
>> ones
>> doing the cleanup work, but afaik they only clean up the db and
>> index for
>> artifacts that are no longer in the filesystem:
>> not-present-remove-db-artifact
>> not-present-remove-db-project
>> not-present-remove-indexed
>>
>> If the problem still persists, do you mind filing an issue for this
>> in
>> jira?
>> :)
>>
>> Thanks,
>> Deng
>>
>>
>>>
>>> Marc Lustig wrote:
>>>>
>>>> Indeed,
>>>> find [REPOSITORY_DIRECTORY_NAME] | xargs touch
>>>> is tremendously faster than
>>>> find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \;
>>>>
>>>>
>>>>
>>>>
>>>> Deng Ching-2 wrote:
>>>>>
>>>>> Hi Marc,
>>>>>
>>>>> Did you touch the contents of the repository? The last modified
>>>>> date
>>> of
>>>>> the
>>>>> artifacts need to be more recent than the last run of the
>>>>> repository
>>>>> scan.
>>>>> You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch
>>>>> *at
>>> the
>>>>> base
>>>>> dir where your repo is located.
>>>>>
>>>>> Thanks,
>>>>> Deng
>>>>>
>>>>> On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig <ml@...>
>>> wrote:
>>>>>
>>>>>>
>>>>>> I forgot to add an excerpt from the log:
>>>>>>
>>>>>> 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor
>>>>>> - Finished repository task:
>>>>>> .\ Scan of internal \.__________________________________________
>>>>>> Repository Dir : /opt/managed_repos/internal
>>>>>> Repository Name : ....
>>>>>> Repository Layout : default
>>>>>> Known Consumers : (7 configured)
>>>>>> auto-rename
>>>>>> metadata-updater
>>>>>> repository-purge
>>>>>> auto-remove
>>>>>> update-db-artifact
>>>>>> create-missing-checksums
>>>>>> index-content
>>>>>> Invalid Consumers : <none>
>>>>>> Duration : 1 Minute 13 Seconds 689 Milliseconds
>>>>>> When Gathered : 6/3/09 5:42 PM
>>>>>> Total File Count : 107263
>>>>>> Avg Time Per File :
>>>>>> ______________________________________________________________
>>>>>> 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Executing task from queue with job name:
>>> database-job:user-requested
>>>>>> 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Task: Updating unprocessed artifacts
>>>>>> 2009-06-03 17:42:55,318 [http-8080-6] INFO
>>>>>> org.apache.maven.archiva.web.action.admin.SchedulerAction -
>>>>>> [ActionMessage]
>>>>>> Your request to update the database has been queued.
>>>>>> 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Task: Updating processed artifacts
>>>>>> 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Finished database task in 85035ms.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Marc Lustig wrote:
>>>>>>>
>>>>>>> We upgraded our test-server to release 1.2.1, removed .index and
>>>>>> .indexer
>>>>>>> from all the repos, ran the Scan for the repos and then the
>>>>>>> Update
>>>>>>> Database.
>>>>>>>
>>>>>>> As a result, before the upgrade we had 500 MB inside in all of
>>>>>>> the
>>>>>> .index
>>>>>>> directories.
>>>>>>> After the upgrade and the repo-scan, the total size is less
>>>>>>> than 1
>>>>>> MB.
>>>>>>> Apparently the repo indexing-process does not work anymore. The
>>>>>>> Search-function does not give any results.
>>>>>>>
>>>>>>> Any idea how to make the repos to get indexed again?
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html
>>>>>> Sent from the archiva-users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p24201482.html
>>> Sent from the archiva-users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p25474649.html
> Sent from the archiva-users mailing list archive at Nabble.com.
>
that helps identify the issue by looking at what data has been
successfully recorded?
On 17/09/2009, at 1:19 AM, Marc Lustig wrote:
>
> Hi,
> I gave this another try by doing the following steps.
>
> 1. deleting repo/.index and repo/.indexer
> 2. touching all files and dirs in repo recursively
> 3. running the repo scan
>
> When we search now for e.g. xmlbeans, we do get a few jar-artifacts
> that
> contain a package-name "xmlbeans". But we do not get the artifact
> whose name
> is xmlbeans.
>
> I have no idea what else to try to make the search-function to work
> again.
> User's here are pretty annoyed about it.
>
> Any suggestion what to do except filing a jira issue?
>
>
>
>
>
> Deng Ching-2 wrote:
>>
>> On Thu, Jun 25, 2009 at 7:26 PM, Marc Lustig <ml@...>
>> wrote:
>>
>>>
>>> On our side, the story is continuing like this.
>>> In the afternoon, I touch the whole repo and let the scan running
>>> and
>>> re-create the database.
>>> After that, it seems like all (physically) existing artifacts can be
>>> found
>>> on the Web-UI. Fine.
>>> The next morning again hundreds of artifacts are disappeared from
>>> the
>>> Web-UI
>>> (but still existing physically). It's so annoying...
>>> I checked the last couple of scans in the log and cannot find
>>> exceptions
>>> for
>>> that Repo concerned.
>>> (However I didn't check all the scans since yesterday afternoon.)
>>
>>
>>>
>>> Anyway, for some reason, some consumer is removing certain
>>> artifacts (or
>>> whole groupId's) from the index. I have no clue why.
>>>
>>> Any suggestion what to do to prevent the removal of artifacts from
>>> the
>>> index-database?
>>>
>>> If we disable the "update-db-artifact" consumer, what undesired
>>> side-effects
>>> may this have?
>>>
>>>
>> Your artifacts won't show up in the webapp browse as this consumer
>> adds
>> the
>> basic artifact info in the db and that's where the artifacts shown
>> in the
>> Browse page is gathered..
>>
>> You can try disabling these consumers in Database page as they are
>> the
>> ones
>> doing the cleanup work, but afaik they only clean up the db and
>> index for
>> artifacts that are no longer in the filesystem:
>> not-present-remove-db-artifact
>> not-present-remove-db-project
>> not-present-remove-indexed
>>
>> If the problem still persists, do you mind filing an issue for this
>> in
>> jira?
>> :)
>>
>> Thanks,
>> Deng
>>
>>
>>>
>>> Marc Lustig wrote:
>>>>
>>>> Indeed,
>>>> find [REPOSITORY_DIRECTORY_NAME] | xargs touch
>>>> is tremendously faster than
>>>> find [REPOSITORY_DIRECTORY_NAME] -exec touch {} \;
>>>>
>>>>
>>>>
>>>>
>>>> Deng Ching-2 wrote:
>>>>>
>>>>> Hi Marc,
>>>>>
>>>>> Did you touch the contents of the repository? The last modified
>>>>> date
>>> of
>>>>> the
>>>>> artifacts need to be more recent than the last run of the
>>>>> repository
>>>>> scan.
>>>>> You can execute *find [REPOSITORY_DIRECTORY_NAME] | xargs touch
>>>>> *at
>>> the
>>>>> base
>>>>> dir where your repo is located.
>>>>>
>>>>> Thanks,
>>>>> Deng
>>>>>
>>>>> On Wed, Jun 3, 2009 at 11:54 PM, Marc Lustig <ml@...>
>>> wrote:
>>>>>
>>>>>>
>>>>>> I forgot to add an excerpt from the log:
>>>>>>
>>>>>> 2009-06-03 17:42:11,554 [pool-2-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaRepositoryScanningTaskExecutor
>>>>>> - Finished repository task:
>>>>>> .\ Scan of internal \.__________________________________________
>>>>>> Repository Dir : /opt/managed_repos/internal
>>>>>> Repository Name : ....
>>>>>> Repository Layout : default
>>>>>> Known Consumers : (7 configured)
>>>>>> auto-rename
>>>>>> metadata-updater
>>>>>> repository-purge
>>>>>> auto-remove
>>>>>> update-db-artifact
>>>>>> create-missing-checksums
>>>>>> index-content
>>>>>> Invalid Consumers : <none>
>>>>>> Duration : 1 Minute 13 Seconds 689 Milliseconds
>>>>>> When Gathered : 6/3/09 5:42 PM
>>>>>> Total File Count : 107263
>>>>>> Avg Time Per File :
>>>>>> ______________________________________________________________
>>>>>> 2009-06-03 17:42:55,317 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Executing task from queue with job name:
>>> database-job:user-requested
>>>>>> 2009-06-03 17:42:55,319 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Task: Updating unprocessed artifacts
>>>>>> 2009-06-03 17:42:55,318 [http-8080-6] INFO
>>>>>> org.apache.maven.archiva.web.action.admin.SchedulerAction -
>>>>>> [ActionMessage]
>>>>>> Your request to update the database has been queued.
>>>>>> 2009-06-03 17:42:55,591 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Task: Updating processed artifacts
>>>>>> 2009-06-03 17:44:20,355 [pool-1-thread-1] INFO
>>>>>>
>>>>>>
>>> org.apache.maven.archiva.scheduled.executors.ArchivaDatabaseUpdateTaskExecutor
>>>>>> - Finished database task in 85035ms.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Marc Lustig wrote:
>>>>>>>
>>>>>>> We upgraded our test-server to release 1.2.1, removed .index and
>>>>>> .indexer
>>>>>>> from all the repos, ran the Scan for the repos and then the
>>>>>>> Update
>>>>>>> Database.
>>>>>>>
>>>>>>> As a result, before the upgrade we had 500 MB inside in all of
>>>>>>> the
>>>>>> .index
>>>>>>> directories.
>>>>>>> After the upgrade and the repo-scan, the total size is less
>>>>>>> than 1
>>>>>> MB.
>>>>>>> Apparently the repo indexing-process does not work anymore. The
>>>>>>> Search-function does not give any results.
>>>>>>>
>>>>>>> Any idea how to make the repos to get indexed again?
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p23854203.html
>>>>>> Sent from the archiva-users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p24201482.html
>>> Sent from the archiva-users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Archiva-1.2.1---repos-won%27t-get-index-tp23854158p25474649.html
> Sent from the archiva-users mailing list archive at Nabble.com.
>
« Return to Thread: Archiva 1.2.1 - repos won't get index
| Free embeddable forum powered by Nabble | Forum Help |
