NFS is no longer work after reboot the server.

[holykim]

I have a server which installed CentOS 5.8 64bit with OpenVZ. I reboot the server due to memory upgrade. After then NFS server doesn't work properly with the error below. I can't figure out how to fix this, please advise.

in /var/log/messages


rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.nametoid/channel' failed: errno 2 (No such file or directory)
rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.idtoname/channel' failed: errno 2 (No such file or directory)

NFS status seems working as below.

service nfs status

rpc.mountd (pid 29524) is running...
nfsd (pid 29521 29520 29519 29518 29517 29516 29515 29514) is running...
rpc.rquotad (pid 29509) is running...

I restarted NFS and it's been done without fail.

Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down NFS services: [ OK ]
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]

However, I found the same error from /var/log/messages and NFS server is not working.

rpc.idmapd[29548]: nfsdreopen: Opening '/proc/net/rpc/nfs4.nametoid/channel' failed: errno 2 (No such file or directory)
rpc.idmapd[29548]: nfsdreopen: Opening '/proc/net/rpc/nfs4.idtoname/channel' failed: errno 2 (No such file or directory)

nfsstat tells me there is no Server nfs v3

Server rpc stats:
calls badcalls badclnt badauth xdrcall
0 0 0 0 0

Client rpc stats:
calls retrans authrefrsh
0 0 0

NFS installed on the Host node, not the containers so I believe it's not a OpenVZ problems. I tried reinstall nfs-utils but it didn't help.

I have tested NFS server on my another servers which has the same CentOS 5.8 and all work fine.

statd, idmapdm rquotad, mountd are loaded.
# ps -ef | grep rpc
rpc 7127 1 0 05:19 ? 00:00:00 portmap
root 7160 155 0 05:19 ? 00:00:00 [rpciod/0]
root 7161 155 0 05:19 ? 00:00:00 [rpciod/1]
root 7162 155 0 05:19 ? 00:00:00 [rpciod/2]
root 7163 155 0 05:19 ? 00:00:00 [rpciod/3]
rpc 10715 9146 0 05:19 ? 00:00:00 portmap
rpc 12615 10295 0 05:20 ? 00:00:00 portmap
rpcuser 23361 1 0 06:00 ? 00:00:00 rpc.statd -p 662 -o 2020
root 29548 1 0 06:25 ? 00:00:00 rpc.idmapd
root 65829 1 0 08:54 ? 00:00:00 rpc.rquotad -p 875
root 65844 1 0 08:54 ? 00:00:00 rpc.mountd -p 892
root 72362 63745 0 09:21 pts/1 00:00:00 grep rpc


Thanks in advance for your help.

Harry

[falko]

What's the output of

getenforce

?

Also, did you check of RAM is ok (e.g. with memtest)?

[holykim]

Thanks Falko

# getenforce
Disabled

I didn't check the memory using a memtest but when the system boot up I haven't got any errors.

I upgraded memory from 14GB to 16GB. I removed two 1GB and installed two 2GB that taken out from the working server, so I did't doubt a memory faulty. Is this possible if the memory faulty caused stop working NFS? Currently all other services are working fine on this server.

Please advise if anything I can do to resolve this issue.

Many Thanks

Harry

P.S The system OS including kernel was up to date using YUM before it's restarted. I tried to reboot the server with the previous kernel but it didn't help. Please help.

[holykim]

I found the broken link in /lib/modules/2.6.18-308.8.2.el5.028stab101.1/build

I installed ovzkernel-devel and now the link is alive. I don't reboot the server yet as it is live situation.

Falko, can I expect the issue to be resolved if I reboot the server? Please advise.

Originally Posted by holykim

Thanks Falko

# getenforce
Disabled

I didn't check the memory using a memtest but when the system boot up I haven't got any errors.

I upgraded memory from 14GB to 16GB. I removed two 1GB and installed two 2GB that taken out from the working server, so I did't doubt a memory faulty. Is this possible if the memory faulty caused stop working NFS? Currently all other services are working fine on this server.

Please advise if anything I can do to resolve this issue.

Many Thanks

Harry

P.S The system OS including kernel was up to date using YUM before it's restarted. I tried to reboot the server with the previous kernel but it didn't help. Please help.

[falko]

Originally Posted by holykim

Falko, can I expect the issue to be resolved if I reboot the server? Please advise.

I can't tell. If you have a rescue system, I'd say it's ok to reboot.

[holykim]

Originally Posted by falko

I can't tell. If you have a rescue system, I'd say it's ok to reboot.

Do yo mean NFS can be resume to normal operation if I reboot the server? Does ovzkernel-dev fix the missing folders below? Currently NFS daemon doesn't create them. Please advise.
'/proc/net/rpc/nfs4.nametoid'
'/proc/net/rpc/nfs4.idtoname'

Updated. ** Reboot the server but nothing changed. Still errors in /var/log/messages as below. Please help!

rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.nametoid/channel' failed: errno 2 (No such file or directory)
rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.idtoname/channel' failed: errno 2 (No such file or directory)

[holykim]

Originally Posted by holykim

Do yo mean NFS can be resume to normal operation if I reboot the server? Does ovzkernel-dev fix the missing folders below? Currently NFS daemon doesn't create them. Please advise.
'/proc/net/rpc/nfs4.nametoid'
'/proc/net/rpc/nfs4.idtoname'

Updated. ** Reboot the server but nothing changed. Still errors in /var/log/messages as below. Please help!

rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.nametoid/channel' failed: errno 2 (No such file or directory)
rpc.idmapd[21461]: nfsdreopen: Opening '/proc/net/rpc/nfs4.idtoname/channel' failed: errno 2 (No such file or directory)

The error message still exist with the rpc.idmapd. However, NFS is now working after reboot the server. What I did is install the ovzkernel-devel and reboot the server. That's it!

I will update if I found the exact reason that caused this issue.

Many thanks!