I/O error reading from mux connection: java.io.EOFException

View: New views
4 Messages — Rating Filter:   Alert me  

I/O error reading from mux connection: java.io.EOFException

by Bryan Thompson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,

I was hoping that someone might shed some light on this exception.  The exception occurs with Jini 2.1, Java 1.6.0_07, NIO enabled, and running under Windows XP Service pack 2 (my laptop).  All services are running on a single host.  The code is a modestly complex JOIN and the exception occurs frequently enough to be expected when the data scale is on the order of 10M rows. I've followed the dicussions concerning some possibly related errors, but, really, there does not appear to be much out there on this.  Related links and stack trace are below.

Thanks in advance,

-bryan


See http://osdir.com/ml/java.sun.javaspaces/2005-09/msg00015.html

See http://osdir.com/ml/java.sun.javaspaces/2005-09/msg00029.html

See http://forums.sun.com/thread.jspa?threadID=5284995 (perhaps post there?)

See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4868432

See http://osdir.com/ml/java.sun.jini/2004-04/msg00274.html

ERROR: 116515 pool-1-thread-54 SYSTAP-BBT.systap.com 6a2d5f17-c36c-4799-ae4c-f61a2e07461a com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:839): java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)

at java.util.concurrent.FutureTask.get(FutureTask.java:83)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:827)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator._hasNext(BlockingBuffer.java:1173)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.hasNext(BlockingBuffer.java:915)

at com.bigdata.service.proxy.ClientAsynchronousIterator.hasNext(ClientAsynchronousIterator.java:514)

at com.bigdata.relation.rule.eval.pipeline.DistributedJoinTask.nextChunk(DistributedJoinTask.java:389)

at com.bigdata.relation.rule.eval.pipeline.JoinTask$BindingSetConsumerTask.call(JoinTask.java:830)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.consumeSources(JoinTask.java:666)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.call(JoinTask.java:449)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.call(JoinTask.java:1)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:619)

Caused by: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at com.bigdata.io.SerializerUtil$1.deserialize(SerializerUtil.java:69)

at com.bigdata.service.proxy.RemoteAsynchronousIteratorImpl$RemoteElementImpl.readExternal(RemoteAsynchronousIteratorImpl.java:201)

at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1792)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1751)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)

at com.sun.jini.jeri.internal.runtime.Util.unmarshalValue(Util.java:221)

at net.jini.jeri.BasicInvocationHandler.unmarshalReturn(BasicInvocationHandler.java:1242)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:825)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659)

at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528)

at $Proxy11.nextElement(Unknown Source)

at com.bigdata.service.proxy.ClientAsynchronousIterator$ReaderTask.call(ClientAsynchronousIterator.java:350)

at com.bigdata.service.proxy.ClientAsynchronousIterator$ReaderTask.call(ClientAsynchronousIterator.java:1)

... 5 more

Caused by: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at com.sun.jini.jeri.internal.mux.Session$MuxInputStream.read(Session.java:943)

at net.jini.jeri.connection.ConnectionManager$Outbound$Input.read(ConnectionManager.java:521)

at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2266)

at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2674)

at java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:2698)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1936)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)

at com.bigdata.io.SerializerUtil$1.deserialize(SerializerUtil.java:65)

... 18 more

Caused by: java.io.EOFException

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO.handleReadReady(SocketChannelConnectionIO.java:384)

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO.access$200(SocketChannelConnectionIO.java:41)

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO$Handler.handleSelection(SocketChannelConnectionIO.java:464)

at com.sun.jini.jeri.internal.runtime.SelectionManager$SelectLoop.run(SelectionManager.java:288)

at com.sun.jini.thread.ThreadPool$Worker.run(ThreadPool.java:136)

... 1 more

 
-------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@...

Parent Message unknown Re: I/O error reading from mux connection: java.io.EOFException

by Bryan Thompson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Michael,
 
I have disabled NIO and tried it.  So far, I have not seen the mux error replicated without NIO.  However, I do get an OOM after a while. I have done some more testing on the laptop and I believe that the root cause is an OOM (both with and without NIO) and that the rest of the errors are cascaded consequences. So, I think that NIO is not "at fault" but just that the problem is revealed in a different manner when NIO is enabled.  I plan to do some multi-machine testing on server platforms and I will see how they behave in contrast to a resource starved laptop. 
 
Thanks,
 
-bryan


From: Michael McConnell [mailto:em.mcconnell@...]
Sent: Tuesday, November 18, 2008 5:40 PM
To: Bryan Thompson
Subject: Re: I/O error reading from mux connection: java.io.EOFException

Just curious - can you disable NIO and see what happens?
 
Also -- saw this posted in one of the archives

 

> It appears that the server (TransientOutriggerImpl) sees the following:
>
> (1) Client invokes server with write()
> (2) Server processes request
> (3) Server attempts to send response to client
> (4) Server reports EOFException because client has closed it's
> connection sometime after (1) but before (3).

Yep, seems that way.

> If the client has initiated
> this, it's not a surprise it sees no exception

I wouldn't expect the client-side remote call to both close the
connection this early and still return normally from the call.

> I'd definitely look to enable logging in the JERI layers on the client
> and see what's happening.



On Tue, Nov 18, 2008 at 1:54 PM, Bryan Thompson <bryan@...> wrote:

Hello,

I was hoping that someone might shed some light on this exception.  The exception occurs with Jini 2.1, Java 1.6.0_07, NIO enabled, and running under Windows XP Service pack 2 (my laptop).  All services are running on a single host.  The code is a modestly complex JOIN and the exception occurs frequently enough to be expected when the data scale is on the order of 10M rows. I've followed the dicussions concerning some possibly related errors, but, really, there does not appear to be much out there on this.  Related links and stack trace are below.

Thanks in advance,

-bryan


See http://osdir.com/ml/java.sun.javaspaces/2005-09/msg00015.html

See http://osdir.com/ml/java.sun.javaspaces/2005-09/msg00029.html

See http://forums.sun.com/thread.jspa?threadID=5284995 (perhaps post there?)

See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4868432

See http://osdir.com/ml/java.sun.jini/2004-04/msg00274.html

ERROR: 116515 pool-1-thread-54 SYSTAP-BBT.systap.com 6a2d5f17-c36c-4799-ae4c-f61a2e07461a com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:839): java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)

at java.util.concurrent.FutureTask.get(FutureTask.java:83)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.checkFuture(BlockingBuffer.java:827)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator._hasNext(BlockingBuffer.java:1173)

at com.bigdata.relation.accesspath.BlockingBuffer$BlockingIterator.hasNext(BlockingBuffer.java:915)

at com.bigdata.service.proxy.ClientAsynchronousIterator.hasNext(ClientAsynchronousIterator.java:514)

at com.bigdata.relation.rule.eval.pipeline.DistributedJoinTask.nextChunk(DistributedJoinTask.java:389)

at com.bigdata.relation.rule.eval.pipeline.JoinTask$BindingSetConsumerTask.call(JoinTask.java:830)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.consumeSources(JoinTask.java:666)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.call(JoinTask.java:449)

at com.bigdata.relation.rule.eval.pipeline.JoinTask.call(JoinTask.java:1)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:619)

Caused by: java.lang.RuntimeException: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at com.bigdata.io.SerializerUtil$1.deserialize(SerializerUtil.java:69)

at com.bigdata.service.proxy.RemoteAsynchronousIteratorImpl$RemoteElementImpl.readExternal(RemoteAsynchronousIteratorImpl.java:201)

at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1792)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1751)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)

at com.sun.jini.jeri.internal.runtime.Util.unmarshalValue(Util.java:221)

at net.jini.jeri.BasicInvocationHandler.unmarshalReturn(BasicInvocationHandler.java:1242)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:825)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659)

at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528)

at $Proxy11.nextElement(Unknown Source)

at com.bigdata.service.proxy.ClientAsynchronousIterator$ReaderTask.call(ClientAsynchronousIterator.java:350)

at com.bigdata.service.proxy.ClientAsynchronousIterator$ReaderTask.call(ClientAsynchronousIterator.java:1)

... 5 more

Caused by: java.io.IOException: I/O error reading from mux connection: java.io.EOFException

at com.sun.jini.jeri.internal.mux.Session$MuxInputStream.read(Session.java:943)

at net.jini.jeri.connection.ConnectionManager$Outbound$Input.read(ConnectionManager.java:521)

at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2266)

at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2674)

at java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:2698)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1936)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)

at com.bigdata.io.SerializerUtil$1.deserialize(SerializerUtil.java:65)

... 18 more

Caused by: java.io.EOFException

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO.handleReadReady(SocketChannelConnectionIO.java:384)

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO.access$200(SocketChannelConnectionIO.java:41)

at com.sun.jini.jeri.internal.mux.SocketChannelConnectionIO$Handler.handleSelection(SocketChannelConnectionIO.java:464)

at com.sun.jini.jeri.internal.runtime.SelectionManager$SelectLoop.run(SelectionManager.java:288)

at com.sun.jini.thread.ThreadPool$Worker.run(ThreadPool.java:136)

... 1 more

 
-------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@...



--
Surely the Germans, who gave us "schadenfreude" and "weltschmertz", have a term for "the uneasy semi-certainty that one's assertion is soundly backed by a long-forgotten source." - unknown

A positive attitude may not solve all your problems, but it will annoy enough people to make it worth the effort.
 - Herm Albright


Slower minds keep right.
- unknown
-------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@...

java spaces and zookeeper

by Bryan Thompson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,
 
I was wondering if anyone has considered whether java spaces can be used to build the same kinds of distributed lock, distributed queue, and master election control structures that are the primary focus of zookeeper[1,2,3].  Zookeeper gives a guarentee of the ordered of events and provides some basic mechanisms for creating nodes within its "space" that are linked to the life span of the client which created them and which are assigned in a sequence that may be used to build up a variety of queue like structures, including distributed locks and master elections. 
 
So, my question is, are there patterns for creating the same kinds of control structures using java spaces?

Thanks,
 
-bryan
 
-------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@...

Re: java spaces and zookeeper

by Guy Korland :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Bryan,

 

GigaSpaces (http://www.gigaspaces.com/wiki) provides a distributed implementation of JavaSpaces.

 

As so it provides means for all the facilities you mentioned including distributed lock (based on javaspaces transaction's model), distributed queue (based on javaspaces notify model + fifo extension) and primary election mechanism (based on jini lookup service).

 

GigaSpaces provides many more facilities to allowing development and deployment of distributed application almost as easy as development of stand alone one.

 

[1] http://www.gigaspaces.com/wiki/display/XAP66/Space+Locking+and+Blocking http://www.gigaspaces.com/wiki/display/XAP66/Pessimistic+Locking

[2] http://www.gigaspaces.com/wiki/display/XAP66/Mule+Queue+Provider http://www.gigaspaces.com/wiki/display/XAP66/Browsing+JMS+Queues

[3] http://www.gigaspaces.com/wiki/display/XAP66/Terminology+-+Data+Grid+Topologies

 

Guy

 


From: Bryan Thompson [mailto:bryan@...]
Sent: Sunday, January 11, 2009 2:47 PM
To: JINI-USERS@...
Subject: java spaces and zookeeper

 

Hello,

 

I was wondering if anyone has considered whether java spaces can be used to build the same kinds of distributed lock, distributed queue, and master election control structures that are the primary focus of zookeeper[1,2,3].  Zookeeper gives a guarentee of the ordered of events and provides some basic mechanisms for creating nodes within its "space" that are linked to the life span of the client which created them and which are assigned in a sequence that may be used to build up a variety of queue like structures, including distributed locks and master elections. 

 

So, my question is, are there patterns for creating the same kinds of control structures using java spaces?


Thanks,

 

-bryan

 

-------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@... -------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to listserv@...