Quantcast
Viewing all articles
Browse latest Browse all 2772

"java.net.ConnectException: Connection refused; No available router to destination"

Hello,

One of the app servers in our QA environment that's configured to process bulk plans is not shutting down. We've 5 app servers and 2 web servers. Among the 5, 2 are configured to be bulk plan servers to process only bulk plans and the other is configured to be a standby. Rest of the 2 app servers do the non-bulk plan activities.

I've scheduled a nightly bounce of the servers. Every night, but for one of the bulk plan app servers, all the other 6 are properly bounced. I'm getting the following error in the shutdown.log.0 :

*****
Failed to connect to t3://valwgpotb002vm.val.vlss.local:7001: Destination unreachable; nested exception is:
java.net.ConnectException: Connection refused; No available router to destination
*****

I can successfully ping and telnet and I've no issues. In the console.log.0, I see a lot of out of memory errors as follows :

*****
INFO | 2013/03/26 08:10:10 | [ERROR][memory ] Throwing OutOfMemory: getNewTla
INFO | 2013/03/26 08:10:10 | [ERROR][memory ] Throwing OutOfMemory: [C, len 9, size 40B
INFO | 2013/03/26 08:10:10 | [ERROR][memory ] Throwing OutOfMemory: getNewTla
*****

And when the shutdown command was issued, here're the error messages logged :

*****
WARNING | 2013/03/26 22:02:58 | Launched Process Forcibly Killed.
WARNING | 2013/03/27 22:03:00 | Launched Process Forcibly Killed.
WARNING | 2013/03/28 22:02:58 | Launched Process Forcibly Killed.
WARNING | 2013/03/29 22:03:00 | Launched Process Forcibly Killed.
WARNING | 2013/03/30 22:02:58 | Launched Process Forcibly Killed.
WARNING | 2013/03/31 22:03:00 | Launched Process Forcibly Killed.
WARNING | 2013/04/01 22:03:00 | Launched Process Forcibly Killed.
*****

This has been going on only for the last 2 weeks, since we upgraded our environment to 6.2.8 from 6.2.2. Since the server couldn't be shutdown, the JVM is up and running and the command to shutdown weblogic server "weblogic.sh stop" hangs.

Any idea what could be the problem ?
Image may be NSFW.
Clik here to view.

Viewing all articles
Browse latest Browse all 2772

Trending Articles