[root@ca20 slava]# grep ca44 /var/log/messages Nov 2 05:58:03 ca44 [ 2761.219452] netconsole: network logging started Nov 2 06:17:41 ca44 [ 3936.889701] Fatal resource shortage: privvmpages, UB 3101. Nov 2 06:17:41 ca44 [ 3936.895288] Fatal resource shortage: privvmpages, UB 3101. Nov 2 06:17:41 ca44 [ 3936.900814] Fatal resource shortage: privvmpages, UB 3101. Nov 2 06:17:41 ca44 [ 3937.228604] Fatal resource shortage: privvmpages, UB 3101. Nov 2 07:05:15 ca44 [ 6786.266133] EXT4-fs (dm-2): Nov 2 07:05:15 ca44 write access unavailable, skipping orphan cleanup Nov 2 07:05:15 ca44 Nov 2 07:05:15 ca44 [ 6786.273392] EXT4-fs (dm-2): Nov 2 07:05:15 ca44 recovery complete Nov 2 07:05:15 ca44 Nov 2 07:05:15 ca44 [ 6786.277890] EXT4-fs (dm-2): Nov 2 07:05:15 ca44 mounted filesystem with ordered data mode Nov 2 07:05:15 ca44 Nov 2 10:34:53 ca44 [19341.866425] CT: 111: started Nov 2 11:17:46 ca44 [21911.719275] netconsole: network logging stopped, interface venet0 unregistered Nov 2 11:17:46 ca44 [21911.764329] netconsole: network logging stopped, interface lo unregistered Nov 2 11:17:46 ca44 [21911.828673] CT: 111: stopped Nov 2 11:21:04 ca44 [22108.984432] CT: 222: started Nov 2 11:35:16 ca44 [22958.383199] netconsole: network logging stopped, interface venet0 unregistered Nov 2 11:35:16 ca44 [22958.438076] netconsole: network logging stopped, interface lo unregistered Nov 2 11:35:16 ca44 [22958.495670] CT: 222: stopped Nov 2 11:41:01 ca44 [23303.641721] CT: 222: started Nov 2 12:31:08 ca44 [26304.923121] netconsole: network logging stopped, interface venet0 unregistered Nov 2 12:31:08 ca44 [26304.971803] netconsole: network logging stopped, interface lo unregistered Nov 2 12:31:08 ca44 [26305.022989] CT: 222: stopped Nov 2 12:32:21 ca44 [26378.920412] CT: 111: started Nov 2 12:59:54 ca44 [28028.604469] possible SYN flooding on port 3306. Sending cookies. Nov 2 13:34:29 ca44 [30099.324906] __ratelimit: 225 callbacks suppressed Nov 2 13:34:29 ca44 [30099.329611] Fatal resource shortage: privvmpages, UB 3101. Nov 2 13:34:29 ca44 [30099.347793] Fatal resource shortage: privvmpages, UB 3101. Nov 2 13:34:29 ca44 [30099.353324] Fatal resource shortage: privvmpages, UB 3101. Nov 2 13:34:29 ca44 [30099.362317] Fatal resource shortage: privvmpages, UB 3101. Nov 2 16:43:17 ca44 [41408.202780] general protection fault: 0000 [#1] Nov 2 16:43:17 ca44 SMP Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.207779] last sysfs file: /sys/devices/virtual/net/venet0/type Nov 2 16:43:17 ca44 [41408.214028] CPU 0 Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.215948] Modules linked in: Nov 2 16:43:17 ca44 dm_snapshot Nov 2 16:43:17 ca44 netconsole Nov 2 16:43:17 ca44 configfs Nov 2 16:43:17 ca44 simfs Nov 2 16:43:17 ca44 vzethdev Nov 2 16:43:17 ca44 vznetdev Nov 2 16:43:17 ca44 vzrst Nov 2 16:43:17 ca44 vzcpt Nov 2 16:43:17 ca44 nfs Nov 2 16:43:17 ca44 fscache Nov 2 16:43:17 ca44 vzdquota Nov 2 16:43:17 ca44 vzmon Nov 2 16:43:17 ca44 vzdev Nov 2 16:43:17 ca44 af_packet Nov 2 16:43:17 ca44 ip6t_REJECT Nov 2 16:43:17 ca44 ip6table_mangle Nov 2 16:43:17 ca44 ipv6 Nov 2 16:43:17 ca44 ip6table_filter Nov 2 16:43:17 ca44 ip6_tables Nov 2 16:43:17 ca44 xt_tcpudp Nov 2 16:43:17 ca44 ipt_REDIRECT Nov 2 16:43:17 ca44 vzevent Nov 2 16:43:17 ca44 nfsd Nov 2 16:43:17 ca44 lockd Nov 2 16:43:17 ca44 nfs_acl Nov 2 16:43:17 ca44 auth_rpcgss Nov 2 16:43:17 ca44 exportfs Nov 2 16:43:17 ca44 sunrpc Nov 2 16:43:17 ca44 8021q Nov 2 16:43:17 ca44 garp Nov 2 16:43:17 ca44 bridge Nov 2 16:43:17 ca44 stp Nov 2 16:43:17 ca44 llc Nov 2 16:43:17 ca44 nf_nat_irc Nov 2 16:43:17 ca44 nf_nat_ftp Nov 2 16:43:17 ca44 nf_conntrack_irc Nov 2 16:43:17 ca44 nf_conntrack_ftp Nov 2 16:43:17 ca44 ipt_REJECT Nov 2 16:43:17 ca44 xt_multiport Nov 2 16:43:17 ca44 xt_recent Nov 2 16:43:17 ca44 xt_owner Nov 2 16:43:17 ca44 xt_state Nov 2 16:43:17 ca44 xt_limit Nov 2 16:43:17 ca44 ipt_LOG Nov 2 16:43:17 ca44 iptable_nat Nov 2 16:43:17 ca44 nf_nat Nov 2 16:43:17 ca44 iptable_mangle Nov 2 16:43:17 ca44 iptable_filter Nov 2 16:43:17 ca44 ip_tables Nov 2 16:43:17 ca44 x_tables Nov 2 16:43:17 ca44 nf_conntrack_ipv4 Nov 2 16:43:17 ca44 nf_conntrack Nov 2 16:43:17 ca44 nf_defrag_ipv4 Nov 2 16:43:17 ca44 nls_utf8 Nov 2 16:43:17 ca44 isofs Nov 2 16:43:17 ca44 loop Nov 2 16:43:17 ca44 usbhid Nov 2 16:43:17 ca44 hid Nov 2 16:43:17 ca44 dm_multipath Nov 2 16:43:17 ca44 scsi_dh Nov 2 16:43:17 ca44 dm_mod Nov 2 16:43:17 ca44 rtc_cmos Nov 2 16:43:17 ca44 rtc_core Nov 2 16:43:17 ca44 power_meter Nov 2 16:43:17 ca44 iTCO_wdt Nov 2 16:43:17 ca44 ehci_hcd Nov 2 16:43:17 ca44 sr_mod Nov 2 16:43:17 ca44 rtc_lib Nov 2 16:43:17 ca44 iTCO_vendor_support Nov 2 16:43:17 ca44 hwmon Nov 2 16:43:17 ca44 evdev Nov 2 16:43:17 ca44 usbcore Nov 2 16:43:17 ca44 cdrom Nov 2 16:43:17 ca44 sg Nov 2 16:43:17 ca44 hed Nov 2 16:43:17 ca44 dcdbas Nov 2 16:43:17 ca44 bnx2 Nov 2 16:43:17 ca44 thermal Nov 2 16:43:17 ca44 processor Nov 2 16:43:17 ca44 ipmi_devintf Nov 2 16:43:17 ca44 ipmi_si Nov 2 16:43:17 ca44 ipmi_msghandler Nov 2 16:43:17 ca44 ipip Nov 2 16:43:17 ca44 tunnel4 Nov 2 16:43:17 ca44 tun Nov 2 16:43:17 ca44 sha1_generic Nov 2 16:43:17 ca44 arc4 Nov 2 16:43:17 ca44 ecb Nov 2 16:43:17 ca44 ppp_mppe Nov 2 16:43:17 ca44 ppp_deflate Nov 2 16:43:17 ca44 zlib_deflate Nov 2 16:43:17 ca44 ppp_async Nov 2 16:43:17 ca44 ppp_generic Nov 2 16:43:17 ca44 slhc Nov 2 16:43:17 ca44 crc_ccitt Nov 2 16:43:17 ca44 button Nov 2 16:43:17 ca44 ext4 Nov 2 16:43:17 ca44 mbcache Nov 2 16:43:17 ca44 jbd2 Nov 2 16:43:17 ca44 crc16 Nov 2 16:43:17 ca44 raid1 Nov 2 16:43:17 ca44 sd_mod Nov 2 16:43:17 ca44 crc_t10dif Nov 2 16:43:17 ca44 ahci Nov 2 16:43:17 ca44 libata Nov 2 16:43:17 ca44 scsi_mod Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.304780] Nov 2 16:43:17 ca44 [41408.306312] Modules linked in: Nov 2 16:43:17 ca44 dm_snapshot Nov 2 16:43:17 ca44 netconsole Nov 2 16:43:17 ca44 configfs Nov 2 16:43:17 ca44 simfs Nov 2 16:43:17 ca44 vzethdev Nov 2 16:43:17 ca44 vznetdev Nov 2 16:43:17 ca44 vzrst Nov 2 16:43:17 ca44 vzcpt Nov 2 16:43:17 ca44 nfs Nov 2 16:43:17 ca44 fscache Nov 2 16:43:17 ca44 vzdquota Nov 2 16:43:17 ca44 vzmon Nov 2 16:43:17 ca44 vzdev Nov 2 16:43:17 ca44 af_packet Nov 2 16:43:17 ca44 ip6t_REJECT Nov 2 16:43:17 ca44 ip6table_mangle Nov 2 16:43:17 ca44 ipv6 Nov 2 16:43:17 ca44 ip6table_filter Nov 2 16:43:17 ca44 ip6_tables Nov 2 16:43:17 ca44 xt_tcpudp Nov 2 16:43:17 ca44 ipt_REDIRECT Nov 2 16:43:17 ca44 vzevent Nov 2 16:43:17 ca44 nfsd Nov 2 16:43:17 ca44 lockd Nov 2 16:43:17 ca44 nfs_acl Nov 2 16:43:17 ca44 auth_rpcgss Nov 2 16:43:17 ca44 exportfs Nov 2 16:43:17 ca44 sunrpc Nov 2 16:43:17 ca44 8021q Nov 2 16:43:17 ca44 garp Nov 2 16:43:17 ca44 bridge Nov 2 16:43:17 ca44 stp Nov 2 16:43:17 ca44 llc Nov 2 16:43:17 ca44 nf_nat_irc Nov 2 16:43:17 ca44 nf_nat_ftp Nov 2 16:43:17 ca44 nf_conntrack_irc Nov 2 16:43:17 ca44 nf_conntrack_ftp Nov 2 16:43:17 ca44 ipt_REJECT Nov 2 16:43:17 ca44 xt_multiport Nov 2 16:43:17 ca44 xt_recent Nov 2 16:43:17 ca44 xt_owner Nov 2 16:43:17 ca44 xt_state Nov 2 16:43:17 ca44 xt_limit Nov 2 16:43:17 ca44 ipt_LOG Nov 2 16:43:17 ca44 iptable_nat Nov 2 16:43:17 ca44 nf_nat Nov 2 16:43:17 ca44 iptable_mangle Nov 2 16:43:17 ca44 iptable_filter Nov 2 16:43:17 ca44 ip_tables Nov 2 16:43:17 ca44 x_tables Nov 2 16:43:17 ca44 nf_conntrack_ipv4 Nov 2 16:43:17 ca44 nf_conntrack Nov 2 16:43:17 ca44 nf_defrag_ipv4 Nov 2 16:43:17 ca44 nls_utf8 Nov 2 16:43:17 ca44 isofs Nov 2 16:43:17 ca44 loop Nov 2 16:43:17 ca44 usbhid Nov 2 16:43:17 ca44 hid Nov 2 16:43:17 ca44 dm_multipath Nov 2 16:43:17 ca44 scsi_dh Nov 2 16:43:17 ca44 dm_mod Nov 2 16:43:17 ca44 rtc_cmos Nov 2 16:43:17 ca44 rtc_core Nov 2 16:43:17 ca44 power_meter Nov 2 16:43:17 ca44 iTCO_wdt Nov 2 16:43:17 ca44 ehci_hcd Nov 2 16:43:17 ca44 sr_mod Nov 2 16:43:17 ca44 rtc_lib Nov 2 16:43:17 ca44 iTCO_vendor_support Nov 2 16:43:17 ca44 hwmon Nov 2 16:43:17 ca44 evdev Nov 2 16:43:17 ca44 usbcore Nov 2 16:43:17 ca44 cdrom Nov 2 16:43:17 ca44 sg Nov 2 16:43:17 ca44 hed Nov 2 16:43:17 ca44 dcdbas Nov 2 16:43:17 ca44 bnx2 Nov 2 16:43:17 ca44 thermal Nov 2 16:43:17 ca44 processor Nov 2 16:43:17 ca44 ipmi_devintf Nov 2 16:43:17 ca44 ipmi_si Nov 2 16:43:17 ca44 ipmi_msghandler Nov 2 16:43:17 ca44 ipip Nov 2 16:43:17 ca44 tunnel4 Nov 2 16:43:17 ca44 tun Nov 2 16:43:17 ca44 sha1_generic Nov 2 16:43:17 ca44 arc4 Nov 2 16:43:17 ca44 ecb Nov 2 16:43:17 ca44 ppp_mppe Nov 2 16:43:17 ca44 ppp_deflate Nov 2 16:43:17 ca44 zlib_deflate Nov 2 16:43:17 ca44 ppp_async Nov 2 16:43:17 ca44 ppp_generic Nov 2 16:43:17 ca44 slhc Nov 2 16:43:17 ca44 crc_ccitt Nov 2 16:43:17 ca44 button Nov 2 16:43:17 ca44 ext4 Nov 2 16:43:17 ca44 mbcache Nov 2 16:43:17 ca44 jbd2 Nov 2 16:43:17 ca44 crc16 Nov 2 16:43:17 ca44 raid1 Nov 2 16:43:17 ca44 sd_mod Nov 2 16:43:17 ca44 crc_t10dif Nov 2 16:43:17 ca44 ahci Nov 2 16:43:17 ca44 libata Nov 2 16:43:17 ca44 scsi_mod Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.395175] Pid: 1015313, comm: httpd Not tainted 2.6.32-ovz-el-alt39 #1 042stab039_5 PowerEdge R210 Nov 2 16:43:17 ca44 [41408.404396] RIP: 0010:[] Nov 2 16:43:17 ca44 [] list_del+0x10/0xa0 Nov 2 16:43:17 ca44 [41408.412163] RSP: 0018:ffff880102bcbc28 EFLAGS: 00010096 Nov 2 16:43:17 ca44 [41408.417502] RAX: dead000000200200 RBX: ffffea0002510528 RCX: 0000000000000001 Nov 2 16:43:17 ca44 [41408.424847] RDX: 00000000ffffffff RSI: 0000000000000003 RDI: ffffea0002510528 Nov 2 16:43:17 ca44 [41408.432087] RBP: ffff880102bcbc38 R08: 0000000000000003 R09: ffffea0002510528 Nov 2 16:43:17 ca44 [41408.439339] R10: ffff88019eeb37c0 R11: ffff880000043d28 R12: ffffea0002510538 Nov 2 16:43:17 ca44 [41408.446619] R13: ffff880000018680 R14: 0000000000000002 R15: 0000000000000000 Nov 2 16:43:17 ca44 [41408.453831] FS: 00007f81881b5760(0000) GS:ffff880030600000(0000) knlGS:0000000000000000 Nov 2 16:43:17 ca44 [41408.461987] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 2 16:43:17 ca44 [41408.467766] CR2: 00007f8188018000 CR3: 000000023bcaf000 CR4: 00000000000006f0 Nov 2 16:43:17 ca44 [41408.475010] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 2 16:43:17 ca44 [41408.482260] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Nov 2 16:43:17 ca44 [41408.489714] Process httpd (pid: 1015313, veid=3101, threadinfo ffff880102bca000, task ffff880083376700) Nov 2 16:43:17 ca44 [41408.499232] Stack: Nov 2 16:43:17 ca44 [41408.501266] 0000000000000001 Nov 2 16:43:17 ca44 ffffea0002510500 Nov 2 16:43:17 ca44 ffff880102bcbcb8 Nov 2 16:43:17 ca44 ffffffff81135c65 Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.508646] <0> Nov 2 16:43:17 ca44 ffff880102bcbd08 Nov 2 16:43:17 ca44 ffff880100000003 Nov 2 16:43:17 ca44 0000000000000001 Nov 2 16:43:17 ca44 ffffea00ffffffff Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.516544] <0> Nov 2 16:43:17 ca44 ffffea0002510528 Nov 2 16:43:17 ca44 ffff880000043cf8 Nov 2 16:43:17 ca44 ffff880000043d28 Nov 2 16:43:17 ca44 0000000000000202 Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.524678] Call Trace: Nov 2 16:43:17 ca44 [41408.527187] [] activate_page+0xe5/0x430 Nov 2 16:43:17 ca44 [41408.532829] [] mark_page_accessed+0x49/0x50 Nov 2 16:43:17 ca44 [41408.538816] [] generic_file_aio_read+0x30f/0x780 Nov 2 16:43:17 ca44 [41408.545192] [] do_sync_read+0xfa/0x140 Nov 2 16:43:17 ca44 [41408.550746] [] ? user_path_at+0x62/0xa0 Nov 2 16:43:17 ca44 [41408.556310] [] ? autoremove_wake_function+0x0/0x40 Nov 2 16:43:17 ca44 [41408.562901] [] ? cp_new_stat+0xe4/0x100 Nov 2 16:43:17 ca44 [41408.568454] [] vfs_read+0xb5/0x1a0 Nov 2 16:43:17 ca44 [41408.573532] [] sys_read+0x51/0x90 Nov 2 16:43:17 ca44 [41408.578580] [] system_call_fastpath+0x16/0x1b Nov 2 16:43:17 ca44 [41408.584648] Code: Nov 2 16:43:17 ca44 89 Nov 2 16:43:17 ca44 95 Nov 2 16:43:17 ca44 fc Nov 2 16:43:17 ca44 fe Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 e9 Nov 2 16:43:17 ca44 ab Nov 2 16:43:17 ca44 fd Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 4c Nov 2 16:43:17 ca44 8b Nov 2 16:43:17 ca44 ad Nov 2 16:43:17 ca44 e8 Nov 2 16:43:17 ca44 fe Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 e9 Nov 2 16:43:17 ca44 db Nov 2 16:43:17 ca44 fd Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 ff Nov 2 16:43:17 ca44 90 Nov 2 16:43:17 ca44 last message repeated 3 times Nov 2 16:43:17 ca44 55 Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 89 Nov 2 16:43:17 ca44 e5 Nov 2 16:43:17 ca44 53 Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 89 Nov 2 16:43:17 ca44 fb Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 83 Nov 2 16:43:17 ca44 ec Nov 2 16:43:17 ca44 08 Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 8b Nov 2 16:43:17 ca44 47 Nov 2 16:43:17 ca44 08 Nov 2 16:43:17 ca44 c> Nov 2 16:43:17 ca44 8b Nov 2 16:43:17 ca44 00 Nov 2 16:43:17 ca44 4c Nov 2 16:43:17 ca44 39 Nov 2 16:43:17 ca44 c7 Nov 2 16:43:17 ca44 75 Nov 2 16:43:17 ca44 39 Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 8b Nov 2 16:43:17 ca44 03 Nov 2 16:43:17 ca44 4c Nov 2 16:43:17 ca44 8b Nov 2 16:43:17 ca44 40 Nov 2 16:43:17 ca44 08 Nov 2 16:43:17 ca44 4c Nov 2 16:43:17 ca44 39 Nov 2 16:43:17 ca44 c3 Nov 2 16:43:17 ca44 75 Nov 2 16:43:17 ca44 4c Nov 2 16:43:17 ca44 48 Nov 2 16:43:17 ca44 Nov 2 16:43:17 ca44 [41408.605100] RIP Nov 2 16:43:17 ca44 [] list_del+0x10/0xa0 Nov 2 16:43:17 ca44 [41408.610516] RSP Nov 2 16:43:18 ca44 [41408.614410] ---[ end trace 3855c742f93e52d0 ]--- Nov 2 16:43:18 ca44 [41408.619109] Kernel panic - not syncing: Fatal exception Nov 2 16:43:18 ca44 [41408.624412] Pid: 1015313, comm: httpd Tainted: G D ---------------- 2.6.32-ovz-el-alt39 #1 Nov 2 16:43:18 ca44 [41408.633650] Call Trace: Nov 2 16:43:18 ca44 [41408.636119] [] ? panic+0x78/0x143 Nov 2 16:43:18 ca44 [41408.641152] [] ? oops_end+0xe4/0x100 Nov 2 16:43:18 ca44 [41408.646542] [] ? die+0x5b/0x90 Nov 2 16:43:18 ca44 [41408.651309] [] ? do_general_protection+0x152/0x160 Nov 2 16:43:18 ca44 [41408.657789] [] ? general_protection+0x25/0x30 Nov 2 16:43:18 ca44 [41408.663941] [] ? list_del+0x10/0xa0 Nov 2 16:43:18 ca44 [41408.669226] [] ? activate_page+0xe5/0x430 Nov 2 16:43:18 ca44 [41408.675014] [] ? mark_page_accessed+0x49/0x50 Nov 2 16:43:18 ca44 [41408.681124] [] ? generic_file_aio_read+0x30f/0x780 Nov 2 16:43:18 ca44 [41408.687768] [] ? do_sync_read+0xfa/0x140 Nov 2 16:43:18 ca44 [41408.693461] [] ? user_path_at+0x62/0xa0 Nov 2 16:43:18 ca44 [41408.699007] [] ? autoremove_wake_function+0x0/0x40 Nov 2 16:43:18 ca44 [41408.705606] [] ? cp_new_stat+0xe4/0x100 Nov 2 16:43:18 ca44 [41408.711204] [] ? vfs_read+0xb5/0x1a0 Nov 2 16:43:18 ca44 [41408.716543] [] ? sys_read+0x51/0x90 Nov 2 16:43:18 ca44 [41408.721751] [] ? system_call_fastpath+0x16/0x1b Nov 2 16:43:19 ca44 [41409.780785] Rebooting in 30 seconds.. [root@ca20 slava]#