|
»
MRM-1097 - too many open file
|
View:
New views
7 Messages
—
Rating Filter:
Alert me
|
 |
MRM-1097 - too many open file
by gwenharold
::
Rate this Message:
Hi,
I've read the issue MRM-1097 and its related issues. This was previously an issue on the CI server that we have and the system administrators set the limits of open file higher than default which was 1024. Version : 1.3.1 Here the steps we've done to reproduced the "too many open files" * set the default of ulimits back to default. * built 2 maven projects with 'clean deploy' as goals - these are deployed in archiva * in archiva did a scan on the repositories. The building and the scanning of the repositories were done simulatenuously. Here the the logs of archiva.log upon reproducing the "too many open files" issue. archiva.log *************************************** 2010-10-01 16:09:57,131 [pool-3-thread-1] ERROR org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor - Error occurred while executing indexing task 'ArtifactIndexingTask [action=ADD, repositoryId=internal, resourceFile=/data/repository/internal/org/codehaus/mojo/taglist-maven-plugin/2.0/taglist-maven-plugin-2.0.jar]': /data/repository/internal/.indexer/_q0i_22w.del (Too many open files) 2010-10-01 16:09:57,131 [pool-3-thread-1] ERROR taskQueueExecutor#indexing - Error executing task org.codehaus.plexus.taskqueue.execution.TaskExecutionException: Error occurred while executing indexing task 'ArtifactIndexingTask [action=ADD, repositoryId=internal, resourceFile=/data/repository/internal/org/codehaus/mojo/taglist-maven-plugin/2.0/taglist-maven-plugin-2.0.jar]' at org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor.executeTask(ArchivaIndexingTaskExecutor.java:188) at org.codehaus.plexus.taskqueue.execution.ThreadedTaskQueueExecutor$ExecutorRunnable$1.run(ThreadedTaskQueueExecutor.java:116) at edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442) at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:987) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:528) at java.lang.Thread.run(Thread.java:595) Caused by: java.io.FileNotFoundException: /data/repository/internal/.indexer/_q0i_22w.del (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212) at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) *************************************** Thanks, Gwen |
|
|
Re: MRM-1097 - too many open file
by Deng Ching-2
::
Rate this Message:
Thanks Gwen!
I found a similar issue in Nexus (https://issues.sonatype.org/browse/NEXUS-3485) and I checked the version of the indexer used and it's 3.0.1. We are currently using version 2.0.0 of the indexer in Archiva 1.3.1 while trunk is using version 2.0.6. As Brett mentioned, upgrading the indexer might fix this problem but we need to verify. -Deng On Fri, Oct 1, 2010 at 6:19 PM, Gwen Harold Autencio <gwenharold20@...> wrote: > Hi, > > I've read the issue MRM-1097 and its related issues. This was previously an issue on the CI server that we have and the system administrators set the limits of open file higher than default which was 1024. > > Version : 1.3.1 > > Here the steps we've done to reproduced the "too many open files" > * set the default of ulimits back to default. > * built 2 maven projects with 'clean deploy' as goals - these are deployed in archiva > * in archiva did a scan on the repositories. > > The building and the scanning of the repositories were done simulatenuously. > > Here the the logs of archiva.log upon reproducing the "too many open files" issue. > > archiva.log > *************************************** > > 2010-10-01 16:09:57,131 [pool-3-thread-1] ERROR org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor - Error occurred while executing indexing task 'ArtifactIndexingTask [action=ADD, repositoryId=internal, resourceFile=/data/repository/internal/org/codehaus/mojo/taglist-maven-plugin/2.0/taglist-maven-plugin-2.0.jar]': /data/repository/internal/.indexer/_q0i_22w.del (Too many open files) > 2010-10-01 16:09:57,131 [pool-3-thread-1] ERROR taskQueueExecutor#indexing - Error executing task > org.codehaus.plexus.taskqueue.execution.TaskExecutionException: Error occurred while executing indexing task 'ArtifactIndexingTask [action=ADD, repositoryId=internal, resourceFile=/data/repository/internal/org/codehaus/mojo/taglist-maven-plugin/2.0/taglist-maven-plugin-2.0.jar]' > at org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor.executeTask(ArchivaIndexingTaskExecutor.java:188) > at org.codehaus.plexus.taskqueue.execution.ThreadedTaskQueueExecutor$ExecutorRunnable$1.run(ThreadedTaskQueueExecutor.java:116) > at edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442) > at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:987) > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:528) > at java.lang.Thread.run(Thread.java:595) > Caused by: java.io.FileNotFoundException: /data/repository/internal/.indexer/_q0i_22w.del (Too many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212) > at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) > at org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) > > *************************************** > > Thanks, > Gwen > > > |
|
|
|
|
|
Re: MRM-1097 - too many open file
by gwenharold
::
Rate this Message:
I've upgraded the nexus indexer to 3.0.1 against the 1.3.x branch in my local machine. And tested it on the server. I experienced the "too many open files" during the first scan of the repository and only occurred to a few files and after scanning it a couple of times and it didn't occurred back.
Here's are some of the logs ************** 2010-10-07 16:22:55,876 [pool-3-thread-1] ERROR org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor - Error occurred while executing indexing task 'ArtifactIndexingTask [action=FINISH, repositoryId=internal, resourceFile=null]': cannot read directory org.apache.lucene.store.FSDirectory@/data/repository/internal/.indexer: list() returned null 2010-10-07 16:22:55,877 [pool-3-thread-1] ERROR org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor - Error occurred while closing context: /data/repository/internal/.indexer/timestamp (Too many open files) 2010-10-07 16:22:55,877 [pool-3-thread-1] ERROR taskQueueExecutor#indexing - Error executing task org.codehaus.plexus.taskqueue.execution.TaskExecutionException: Error occurred while closing context: /data/repository/internal/.indexer/timestamp (Too many open files) at org.apache.maven.archiva.scheduled.executors.ArchivaIndexingTaskExecutor.executeTask(ArchivaIndexingTaskExecutor.java:112) at org.codehaus.plexus.taskqueue.execution.ThreadedTaskQueueExecutor$ExecutorRunnable$1.run(ThreadedTaskQueueExecutor.java:116) at edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442) at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:987) at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:528) at java.lang.Thread.run(Thread.java:595) ******************** Thanks, Gwen --- On Mon, 10/4/10, Gwen Harold Autencio <gwenharold20@...> wrote: > From: Gwen Harold Autencio <gwenharold20@...> > Subject: Re: MRM-1097 - too many open file > To: dev@... > Date: Monday, October 4, 2010, 10:29 PM > I'm trying to upgrade the nexus > indexer to version 3.0.1 in my local to archiva, to verify > if the "too many open files" would still occur but I haven't > successful upgrade it yet locally to test it > I'm getting failed test in NexusRepositorySearchTest. I've > already tried to remove "plexus-container-default" from the > trunk pom and added it in archiva-indexer module with a > scope test on it. > > Any ideas on this error ? > > This is the error I'm getting > > ***************************** > testQuickSearch(org.apache.archiva.indexer.search.NexusRepositorySearchTest) > Time elapsed: 1.118 sec <<< ERROR! > org.springframework.beans.factory.BeanCreationException: > Error creating bean with name 'nexusSearch' defined in URL > [file:/home/gwen/Desktop/archiva/archiva-modules/archiva-base/archiva-indexer/target/classes/META-INF/spring-context.xml]: > Cannot resolve reference to bean 'nexusIndexer' while > setting constructor argument; nested exception is > org.springframework.beans.factory.BeanCreationException: > Error creating bean with name 'nexusIndexer': FactoryBean > threw exception on object creation; nested exception is > org.springframework.beans.factory.BeanCreationException: > Error creating bean with name 'queryCreator': FactoryBean > threw exception on object creation; nested exception is > org.springframework.beans.factory.NoSuchBeanDefinitionException: > No bean named 'logger' is defined > at > org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:275) > at > org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:104) > at > org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:495) > ***************************** > > Thanks, > Gwen > > --- On Sun, 10/3/10, Deng Ching <oching@...> > wrote: > > From: Deng Ching <oching@...> > Subject: Re: MRM-1097 - too many open file > To: dev@... > Date: Sunday, October 3, 2010, 12:02 PM > > Thanks Gwen! > > I found a similar issue in Nexus > (https://issues.sonatype.org/browse/NEXUS-3485) and I > checked the > version of the indexer used and it's 3.0.1. We are > currently using > version 2.0.0 of the indexer in Archiva 1.3.1 while trunk > is using > version 2.0.6. As Brett mentioned, upgrading the indexer > might fix > this problem but we need to verify. > > -Deng > > > > > > |
|
|
Re: MRM-1097 - too many open file
by Wendy Smoak-3
::
Rate this Message:
On Thu, Oct 7, 2010 at 11:35 AM, Gwen Harold Autencio
<gwenharold20@...> wrote: > I've upgraded the nexus indexer to 3.0.1 against the 1.3.x branch in my local machine. And tested it on the server. I experienced the "too many open files" during the first scan of the repository and only occurred to a few files and after scanning it a couple of times and it didn't occurred back. How does that compare to before? It sounds like upgrading the indexer did not fix the problem. Subsequent scans aren't going to find any files unless you've added new content, right? -- Wendy |
|
|
Re: MRM-1097 - too many open file
by brettporter
::
Rate this Message:
In addition to Wendy's comment, it's worth checking what version of Lucene is being used and if it's the same one expected by the indexer.
On 08/10/2010, at 2:54 AM, Wendy Smoak wrote: > On Thu, Oct 7, 2010 at 11:35 AM, Gwen Harold Autencio > <gwenharold20@...> wrote: >> I've upgraded the nexus indexer to 3.0.1 against the 1.3.x branch in my local machine. And tested it on the server. I experienced the "too many open files" during the first scan of the repository and only occurred to a few files and after scanning it a couple of times and it didn't occurred back. > > How does that compare to before? It sounds like upgrading the indexer > did not fix the problem. > > Subsequent scans aren't going to find any files unless you've added > new content, right? > > -- > Wendy -- Brett Porter brett@... http://brettporter.wordpress.com/ |
|
|
Re: MRM-1097 - too many open file
by gwenharold
::
Rate this Message:
Yes. adding contents while doing a scan will produce the error.
Have just tried lucene 2.4.1 in archiva same version of the indexer, error still occurs. --- On Fri, 10/8/10, Brett Porter <brett@...> wrote: > From: Brett Porter <brett@...> > Subject: Re: MRM-1097 - too many open file > To: dev@... > Date: Friday, October 8, 2010, 8:34 AM > In addition to Wendy's comment, it's > worth checking what version of Lucene is being used and if > it's the same one expected by the indexer. > > On 08/10/2010, at 2:54 AM, Wendy Smoak wrote: > > > On Thu, Oct 7, 2010 at 11:35 AM, Gwen Harold Autencio > > <gwenharold20@...> > wrote: > >> I've upgraded the nexus indexer to 3.0.1 against > the 1.3.x branch in my local machine. And tested it on the > server. I experienced the "too many open files" during the > first scan of the repository and only occurred to a few > files and after scanning it a couple of times and it > didn't occurred back. > > > > How does that compare to before? It sounds like > upgrading the indexer > > did not fix the problem. > > > > Subsequent scans aren't going to find any files unless > you've added > > new content, right? > > > > -- > > Wendy > > -- > Brett Porter > brett@... > http://brettporter.wordpress.com/ > > > > > |
| Free embeddable forum powered by Nabble | Forum Help |
