Mine

2014-01-18T13:48:27.853241+00:00 bl-cassoop-p11 kernel: [74679.844989] Memory cgroup out of memory: Kill process 36617 (java) score 27 or sacrifice child
2014-01-18T13:48:51.796175+00:00 bl-cassoop-p11 kernel: [74698.770893] ------------[ cut here ]------------
2014-01-18T13:48:51.796195+00:00 bl-cassoop-p11 kernel: [74698.770904] WARNING: CPU: 15 PID: 36402 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:48:51.796198+00:00 bl-cassoop-p11 kernel: [74698.770905] Watchdog detected hard LOCKUP on cpu 15
2014-01-18T13:48:51.796202+00:00 bl-cassoop-p11 kernel: [74698.770907] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:51.796205+00:00 bl-cassoop-p11 kernel: [74698.770928] CPU: 15 PID: 36402 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:51.796207+00:00 bl-cassoop-p11 kernel: [74698.770929] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:51.796208+00:00 bl-cassoop-p11 kernel: [74698.770931]  00000000000000f5 ffff88301f2e7ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:48:51.796210+00:00 bl-cassoop-p11 kernel: [74698.770936]  ffff88301f2e7bf8 ffff88301f2e7be8 ffffffff810653ac 0000000000000000
2014-01-18T13:48:51.796211+00:00 bl-cassoop-p11 kernel: [74698.770938]  ffff882ffa548000 0000000000000000 ffff88301f2e7d20 0000000000000000
2014-01-18T13:48:51.796212+00:00 bl-cassoop-p11 kernel: [74698.770941] Call Trace:
2014-01-18T13:48:51.796215+00:00 bl-cassoop-p11 kernel: [74698.770943]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:48:51.796218+00:00 bl-cassoop-p11 kernel: [74698.770955]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:48:51.796220+00:00 bl-cassoop-p11 kernel: [74698.770958]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:48:51.796223+00:00 bl-cassoop-p11 kernel: [74698.770960]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:48:51.796251+00:00 bl-cassoop-p11 kernel: [74698.770965]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:48:51.796254+00:00 bl-cassoop-p11 kernel: [74698.770970]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:48:51.796255+00:00 bl-cassoop-p11 kernel: [74698.770973]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:48:51.796257+00:00 bl-cassoop-p11 kernel: [74698.770977]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:48:51.796260+00:00 bl-cassoop-p11 kernel: [74698.770982]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:48:51.796262+00:00 bl-cassoop-p11 kernel: [74698.770987]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:48:51.796263+00:00 bl-cassoop-p11 kernel: [74698.770992]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:48:51.796265+00:00 bl-cassoop-p11 kernel: [74698.770995]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:48:51.796278+00:00 bl-cassoop-p11 kernel: [74698.770998]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:48:51.796280+00:00 bl-cassoop-p11 kernel: [74698.771000]  [<ffffffff81748f39>] default_do_nmi+0xe9/0x240
2014-01-18T13:48:51.796281+00:00 bl-cassoop-p11 kernel: [74698.771003]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:48:51.796284+00:00 bl-cassoop-p11 kernel: [74698.771006]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:48:51.796286+00:00 bl-cassoop-p11 kernel: [74698.771011]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:48:51.796287+00:00 bl-cassoop-p11 kernel: [74698.771014]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:48:51.796289+00:00 bl-cassoop-p11 kernel: [74698.771016]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:48:51.796291+00:00 bl-cassoop-p11 kernel: [74698.771017]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:48:51.796292+00:00 bl-cassoop-p11 kernel: [74698.771023]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:48:51.796294+00:00 bl-cassoop-p11 kernel: [74698.771026]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:48:51.796296+00:00 bl-cassoop-p11 kernel: [74698.771028]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:48:51.796298+00:00 bl-cassoop-p11 kernel: [74698.771034]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:48:51.796300+00:00 bl-cassoop-p11 kernel: [74698.771036]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:48:51.796301+00:00 bl-cassoop-p11 kernel: [74698.771039]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:48:51.796303+00:00 bl-cassoop-p11 kernel: [74698.771045]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:48:51.796305+00:00 bl-cassoop-p11 kernel: [74698.771049]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:48:51.796306+00:00 bl-cassoop-p11 kernel: [74698.771055]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:48:51.796308+00:00 bl-cassoop-p11 kernel: [74698.771057]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:48:51.796309+00:00 bl-cassoop-p11 kernel: [74698.771060]  [<ffffffff81747c7c>] retint_signal+0x48/0x8c
2014-01-18T13:48:51.796311+00:00 bl-cassoop-p11 kernel: [74698.771061] ---[ end trace 51e5206791572efe ]---
2014-01-18T13:48:51.796313+00:00 bl-cassoop-p11 kernel: [74703.802920] BUG: soft lockup - CPU#30 stuck for 23s! [java:36244]
2014-01-18T13:48:51.796317+00:00 bl-cassoop-p11 kernel: [74703.809108] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:51.796319+00:00 bl-cassoop-p11 kernel: [74703.809136] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:51.796320+00:00 bl-cassoop-p11 kernel: [74703.809137] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:51.796322+00:00 bl-cassoop-p11 kernel: [74703.809140] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:48:51.796362+00:00 bl-cassoop-p11 kernel: [74703.809142] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:48:51.796364+00:00 bl-cassoop-p11 kernel: [74703.809148] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:48:51.796366+00:00 bl-cassoop-p11 kernel: [74703.809150] RAX: 0000000000000286 RBX: ffff88187fffcb08 RCX: 0000000000000001
2014-01-18T13:48:51.796367+00:00 bl-cassoop-p11 kernel: [74703.809151] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:48:51.796369+00:00 bl-cassoop-p11 kernel: [74703.809153] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:48:51.796370+00:00 bl-cassoop-p11 kernel: [74703.809154] R10: 00000000000035de R11: 0000000000000001 R12: 0000000000000000
2014-01-18T13:48:51.796372+00:00 bl-cassoop-p11 kernel: [74703.809156] R13: ffff88187fffcb00 R14: ffff88187fffcb08 R15: ffffffff00000000
2014-01-18T13:48:51.796374+00:00 bl-cassoop-p11 kernel: [74703.809158] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:48:51.796375+00:00 bl-cassoop-p11 kernel: [74703.809160] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:51.796376+00:00 bl-cassoop-p11 kernel: [74703.809162] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:48:51.796378+00:00 bl-cassoop-p11 kernel: [74703.809164] Stack:
2014-01-18T13:48:51.796379+00:00 bl-cassoop-p11 kernel: [74703.809165]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:48:51.796381+00:00 bl-cassoop-p11 kernel: [74703.809172]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:48:51.796382+00:00 bl-cassoop-p11 kernel: [74703.809176]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:48:51.796384+00:00 bl-cassoop-p11 kernel: [74703.809181] Call Trace:
2014-01-18T13:48:51.796386+00:00 bl-cassoop-p11 kernel: [74703.809188]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:51.796388+00:00 bl-cassoop-p11 kernel: [74703.809191]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:48:51.796390+00:00 bl-cassoop-p11 kernel: [74703.809195]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:51.796391+00:00 bl-cassoop-p11 kernel: [74703.809198]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:51.796393+00:00 bl-cassoop-p11 kernel: [74703.809201]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:48:51.796395+00:00 bl-cassoop-p11 kernel: [74703.809206]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:48:51.796397+00:00 bl-cassoop-p11 kernel: [74703.809210]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:51.796399+00:00 bl-cassoop-p11 kernel: [74703.809214]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:48:51.796401+00:00 bl-cassoop-p11 kernel: [74703.809220]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:48:51.796402+00:00 bl-cassoop-p11 kernel: [74703.809223]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:48:51.796404+00:00 bl-cassoop-p11 kernel: [74703.809226]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:48:51.796406+00:00 bl-cassoop-p11 kernel: [74703.809231]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:48:51.796407+00:00 bl-cassoop-p11 kernel: [74703.809235]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:48:51.796409+00:00 bl-cassoop-p11 kernel: [74703.809238]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:48:51.796411+00:00 bl-cassoop-p11 kernel: [74703.809242]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:48:51.796413+00:00 bl-cassoop-p11 kernel: [74703.809247]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:48:51.796414+00:00 bl-cassoop-p11 kernel: [74703.809251]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:48:51.796416+00:00 bl-cassoop-p11 kernel: [74703.809254]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:48:51.796417+00:00 bl-cassoop-p11 kernel: [74703.809258]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:48:51.796419+00:00 bl-cassoop-p11 kernel: [74703.809263]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:48:51.796420+00:00 bl-cassoop-p11 kernel: [74703.809266]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:48:51.796422+00:00 bl-cassoop-p11 kernel: [74703.809271]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:48:51.796424+00:00 bl-cassoop-p11 kernel: [74703.809275]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:48:51.796426+00:00 bl-cassoop-p11 kernel: [74703.809278]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:48:51.796427+00:00 bl-cassoop-p11 kernel: [74703.809284]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:48:51.796429+00:00 bl-cassoop-p11 kernel: [74703.809289]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:48:51.796430+00:00 bl-cassoop-p11 kernel: [74703.809292]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:48:51.796432+00:00 bl-cassoop-p11 kernel: [74703.809297]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:48:51.796435+00:00 bl-cassoop-p11 kernel: [74703.809299] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:48:55.443947+00:00 bl-cassoop-p11 kernel: [74707.452254] BUG: soft lockup - CPU#0 stuck for 22s! [java:42506]
2014-01-18T13:48:55.443956+00:00 bl-cassoop-p11 kernel: [74707.458327] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.443959+00:00 bl-cassoop-p11 kernel: [74707.458351] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.443961+00:00 bl-cassoop-p11 kernel: [74707.458353] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.443962+00:00 bl-cassoop-p11 kernel: [74707.458355] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:48:55.443965+00:00 bl-cassoop-p11 kernel: [74707.458356] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:48:55.443966+00:00 bl-cassoop-p11 kernel: [74707.458361] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:48:55.443967+00:00 bl-cassoop-p11 kernel: [74707.458363] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa97868
2014-01-18T13:48:55.443969+00:00 bl-cassoop-p11 kernel: [74707.458364] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.443970+00:00 bl-cassoop-p11 kernel: [74707.458366] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:48:55.443972+00:00 bl-cassoop-p11 kernel: [74707.458368] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:48:55.443973+00:00 bl-cassoop-p11 kernel: [74707.458369] R13: ffff880045ccfc48 R14: 0000000000000292 R15: ffff880045ccfc78
2014-01-18T13:48:55.443975+00:00 bl-cassoop-p11 kernel: [74707.458371] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.443976+00:00 bl-cassoop-p11 kernel: [74707.458373] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.443977+00:00 bl-cassoop-p11 kernel: [74707.458375] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:48:55.443978+00:00 bl-cassoop-p11 kernel: [74707.458376] Stack:
2014-01-18T13:48:55.443980+00:00 bl-cassoop-p11 kernel: [74707.458378]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:48:55.443981+00:00 bl-cassoop-p11 kernel: [74707.458384]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:48:55.443983+00:00 bl-cassoop-p11 kernel: [74707.458388]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:48:55.443984+00:00 bl-cassoop-p11 kernel: [74707.458393] Call Trace:
2014-01-18T13:48:55.443985+00:00 bl-cassoop-p11 kernel: [74707.458398]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.443987+00:00 bl-cassoop-p11 kernel: [74707.458402]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:48:55.443989+00:00 bl-cassoop-p11 kernel: [74707.458406]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:48:55.443990+00:00 bl-cassoop-p11 kernel: [74707.458410]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:48:55.443992+00:00 bl-cassoop-p11 kernel: [74707.458413]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:48:55.443993+00:00 bl-cassoop-p11 kernel: [74707.458417]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:48:55.443995+00:00 bl-cassoop-p11 kernel: [74707.458420]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:48:55.443996+00:00 bl-cassoop-p11 kernel: [74707.458424]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:48:55.443998+00:00 bl-cassoop-p11 kernel: [74707.458428]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:48:55.444000+00:00 bl-cassoop-p11 kernel: [74707.458429] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:48:55.555846+00:00 bl-cassoop-p11 kernel: [74707.564293] BUG: soft lockup - CPU#10 stuck for 22s! [java:42435]
2014-01-18T13:48:55.555853+00:00 bl-cassoop-p11 kernel: [74707.570256] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.555872+00:00 bl-cassoop-p11 kernel: [74707.570273] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.555874+00:00 bl-cassoop-p11 kernel: [74707.570274] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.555876+00:00 bl-cassoop-p11 kernel: [74707.570276] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:48:55.555878+00:00 bl-cassoop-p11 kernel: [74707.570277] RIP: 0010:[<ffffffff81731061>]  [<ffffffff81731061>] oom_kill_process.part.6+0xa7/0x2cf
2014-01-18T13:48:55.555879+00:00 bl-cassoop-p11 kernel: [74707.570282] RSP: 0000:ffff8819e9a81a28  EFLAGS: 00000287
2014-01-18T13:48:55.555881+00:00 bl-cassoop-p11 kernel: [74707.570283] RAX: ffff882ff0e31b00 RBX: 00000000000000d0 RCX: 0000000000000006
2014-01-18T13:48:55.555882+00:00 bl-cassoop-p11 kernel: [74707.570285] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffffffff81c06040
2014-01-18T13:48:55.555884+00:00 bl-cassoop-p11 kernel: [74707.570286] RBP: ffff8819e9a81a88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:48:55.555885+00:00 bl-cassoop-p11 kernel: [74707.570287] R10: 0000000000000e32 R11: 0000000000005000 R12: ffff882ff044e3f0
2014-01-18T13:48:55.555886+00:00 bl-cassoop-p11 kernel: [74707.570288] R13: 0000000000000005 R14: 0000000000000246 R15: 0000000028e74000
2014-01-18T13:48:55.555888+00:00 bl-cassoop-p11 kernel: [74707.570289] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.555889+00:00 bl-cassoop-p11 kernel: [74707.570291] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.555891+00:00 bl-cassoop-p11 kernel: [74707.570292] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:48:55.555891+00:00 bl-cassoop-p11 kernel: [74707.570293] Stack:
2014-01-18T13:48:55.555893+00:00 bl-cassoop-p11 kernel: [74707.570294]  ffff8819e9a81a38 ffff882ff044ddc0 000000000114eccd ffffc90028e74000
2014-01-18T13:48:55.555894+00:00 bl-cassoop-p11 kernel: [74707.570297]  ffff8819e9a81a88 ffff882ff0e31b00 ffff882ff5c55dc0 000000000114eccd
2014-01-18T13:48:55.555896+00:00 bl-cassoop-p11 kernel: [74707.570300]  0000000000000000 ffffc90028e74000 ffff882ff0b86a00 ffff882ff044ddc0
2014-01-18T13:48:55.555897+00:00 bl-cassoop-p11 kernel: [74707.570303] Call Trace:
2014-01-18T13:48:55.555898+00:00 bl-cassoop-p11 kernel: [74707.570308]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:48:55.555900+00:00 bl-cassoop-p11 kernel: [74707.570310]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:48:55.555902+00:00 bl-cassoop-p11 kernel: [74707.570313]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:48:55.555903+00:00 bl-cassoop-p11 kernel: [74707.570316]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:48:55.555905+00:00 bl-cassoop-p11 kernel: [74707.570320]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:48:55.555906+00:00 bl-cassoop-p11 kernel: [74707.570323]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:48:55.555908+00:00 bl-cassoop-p11 kernel: [74707.570325]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:48:55.555910+00:00 bl-cassoop-p11 kernel: [74707.570328]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:48:55.555912+00:00 bl-cassoop-p11 kernel: [74707.570331]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:48:55.555913+00:00 bl-cassoop-p11 kernel: [74707.570334]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:48:55.555915+00:00 bl-cassoop-p11 kernel: [74707.570336]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:48:55.555916+00:00 bl-cassoop-p11 kernel: [74707.570339]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:48:55.555918+00:00 bl-cassoop-p11 kernel: [74707.570341]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:48:55.555919+00:00 bl-cassoop-p11 kernel: [74707.570343]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:48:55.555921+00:00 bl-cassoop-p11 kernel: [74707.570346]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:48:55.555922+00:00 bl-cassoop-p11 kernel: [74707.570349]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:48:55.555924+00:00 bl-cassoop-p11 kernel: [74707.570352]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:48:55.555925+00:00 bl-cassoop-p11 kernel: [74707.570355]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:48:55.555927+00:00 bl-cassoop-p11 kernel: [74707.570359]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:48:55.555928+00:00 bl-cassoop-p11 kernel: [74707.570361]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:48:55.555930+00:00 bl-cassoop-p11 kernel: [74707.570364]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:48:55.555931+00:00 bl-cassoop-p11 kernel: [74707.570366]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:48:55.555933+00:00 bl-cassoop-p11 kernel: [74707.570369]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:48:55.555935+00:00 bl-cassoop-p11 kernel: [74707.570371] Code: 10 c3 b0 81 31 c0 e8 d1 e5 ff ff 48 89 df e8 e7 ff 91 ff 66 90 48 c7 c7 40 60 c0 81 4d 89 fe 45 31 ed e8 63 65 01 00 4c 89 7d a8 <49> 8b 9e 00 03 00 00 4d 8d a6 00 03 00 00 48 81 eb 10 03 00 00
2014-01-18T13:48:55.579663+00:00 bl-cassoop-p11 kernel: [74707.588301] BUG: soft lockup - CPU#12 stuck for 22s! [jsvc:7242]
2014-01-18T13:48:55.579670+00:00 bl-cassoop-p11 kernel: [74707.594103] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.579673+00:00 bl-cassoop-p11 kernel: [74707.594123] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.579675+00:00 bl-cassoop-p11 kernel: [74707.594124] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.579676+00:00 bl-cassoop-p11 kernel: [74707.594126] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:48:55.579679+00:00 bl-cassoop-p11 kernel: [74707.594127] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:48:55.579680+00:00 bl-cassoop-p11 kernel: [74707.594130] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:48:55.579681+00:00 bl-cassoop-p11 kernel: [74707.594132] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa38680
2014-01-18T13:48:55.579683+00:00 bl-cassoop-p11 kernel: [74707.594133] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.579684+00:00 bl-cassoop-p11 kernel: [74707.594134] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:48:55.579686+00:00 bl-cassoop-p11 kernel: [74707.594135] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff8817f37ca000
2014-01-18T13:48:55.579687+00:00 bl-cassoop-p11 kernel: [74707.594136] R13: 0000000000000001 R14: 0000000000000292 R15: ffff8817f8e45938
2014-01-18T13:48:55.579689+00:00 bl-cassoop-p11 kernel: [74707.594138] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.579690+00:00 bl-cassoop-p11 kernel: [74707.594139] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.579691+00:00 bl-cassoop-p11 kernel: [74707.594140] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:48:55.579692+00:00 bl-cassoop-p11 kernel: [74707.594141] Stack:
2014-01-18T13:48:55.579694+00:00 bl-cassoop-p11 kernel: [74707.594142]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:48:55.579695+00:00 bl-cassoop-p11 kernel: [74707.594146]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:48:55.579697+00:00 bl-cassoop-p11 kernel: [74707.594149]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:48:55.579698+00:00 bl-cassoop-p11 kernel: [74707.594152] Call Trace:
2014-01-18T13:48:55.579699+00:00 bl-cassoop-p11 kernel: [74707.594155]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.579701+00:00 bl-cassoop-p11 kernel: [74707.594158]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.579702+00:00 bl-cassoop-p11 kernel: [74707.594161]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:48:55.579704+00:00 bl-cassoop-p11 kernel: [74707.594164]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:48:55.579706+00:00 bl-cassoop-p11 kernel: [74707.594166]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:48:55.579708+00:00 bl-cassoop-p11 kernel: [74707.594169]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:48:55.579709+00:00 bl-cassoop-p11 kernel: [74707.594171]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:48:55.579711+00:00 bl-cassoop-p11 kernel: [74707.594174]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:48:55.579712+00:00 bl-cassoop-p11 kernel: [74707.594176]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:48:55.579714+00:00 bl-cassoop-p11 kernel: [74707.594178]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:48:55.579716+00:00 bl-cassoop-p11 kernel: [74707.594180]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:48:55.579717+00:00 bl-cassoop-p11 kernel: [74707.594182]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:48:55.579719+00:00 bl-cassoop-p11 kernel: [74707.594184]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:48:55.579720+00:00 bl-cassoop-p11 kernel: [74707.594189]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:48:55.579722+00:00 bl-cassoop-p11 kernel: [74707.594191]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:48:55.579723+00:00 bl-cassoop-p11 kernel: [74707.594193]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:48:55.579725+00:00 bl-cassoop-p11 kernel: [74707.594196]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:48:55.579726+00:00 bl-cassoop-p11 kernel: [74707.594199]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:48:55.579729+00:00 bl-cassoop-p11 kernel: [74707.594200] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:48:55.591541+00:00 bl-cassoop-p11 kernel: [74707.600309] BUG: soft lockup - CPU#13 stuck for 22s! [kswapd1:284]
2014-01-18T13:48:55.591548+00:00 bl-cassoop-p11 kernel: [74707.605971] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.591550+00:00 bl-cassoop-p11 kernel: [74707.605987] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.591552+00:00 bl-cassoop-p11 kernel: [74707.605988] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.591553+00:00 bl-cassoop-p11 kernel: [74707.605990] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:48:55.591556+00:00 bl-cassoop-p11 kernel: [74707.605991] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:48:55.591557+00:00 bl-cassoop-p11 kernel: [74707.605994] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:48:55.591558+00:00 bl-cassoop-p11 kernel: [74707.605996] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa386a8
2014-01-18T13:48:55.591560+00:00 bl-cassoop-p11 kernel: [74707.605997] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.591561+00:00 bl-cassoop-p11 kernel: [74707.605998] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:48:55.591563+00:00 bl-cassoop-p11 kernel: [74707.605999] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000009
2014-01-18T13:48:55.591564+00:00 bl-cassoop-p11 kernel: [74707.606000] R13: ffffffff8105ae90 R14: 0000000000000292 R15: ffff882ff8363748
2014-01-18T13:48:55.591566+00:00 bl-cassoop-p11 kernel: [74707.606001] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.591567+00:00 bl-cassoop-p11 kernel: [74707.606002] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.591568+00:00 bl-cassoop-p11 kernel: [74707.606003] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:48:55.591569+00:00 bl-cassoop-p11 kernel: [74707.606004] Stack:
2014-01-18T13:48:55.591571+00:00 bl-cassoop-p11 kernel: [74707.606005]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:48:55.591573+00:00 bl-cassoop-p11 kernel: [74707.606009]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:48:55.591574+00:00 bl-cassoop-p11 kernel: [74707.606012]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:48:55.591575+00:00 bl-cassoop-p11 kernel: [74707.606015] Call Trace:
2014-01-18T13:48:55.591577+00:00 bl-cassoop-p11 kernel: [74707.606020]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.591578+00:00 bl-cassoop-p11 kernel: [74707.606023]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.591580+00:00 bl-cassoop-p11 kernel: [74707.606026]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:48:55.591581+00:00 bl-cassoop-p11 kernel: [74707.606029]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:48:55.591583+00:00 bl-cassoop-p11 kernel: [74707.606032]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:48:55.591584+00:00 bl-cassoop-p11 kernel: [74707.606035]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:48:55.591586+00:00 bl-cassoop-p11 kernel: [74707.606039]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:48:55.591588+00:00 bl-cassoop-p11 kernel: [74707.606042]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:48:55.591589+00:00 bl-cassoop-p11 kernel: [74707.606045]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:48:55.591591+00:00 bl-cassoop-p11 kernel: [74707.606048]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:48:55.591592+00:00 bl-cassoop-p11 kernel: [74707.606052]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:48:55.591594+00:00 bl-cassoop-p11 kernel: [74707.606054]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:48:55.591595+00:00 bl-cassoop-p11 kernel: [74707.606057]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:48:55.591597+00:00 bl-cassoop-p11 kernel: [74707.606059]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:48:55.591599+00:00 bl-cassoop-p11 kernel: [74707.606062]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:48:55.591600+00:00 bl-cassoop-p11 kernel: [74707.606066]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:48:55.591602+00:00 bl-cassoop-p11 kernel: [74707.606069]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:48:55.591603+00:00 bl-cassoop-p11 kernel: [74707.606071]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:48:55.591605+00:00 bl-cassoop-p11 kernel: [74707.606075]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:48:55.591606+00:00 bl-cassoop-p11 kernel: [74707.606078]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.591608+00:00 bl-cassoop-p11 kernel: [74707.606081]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:48:55.591609+00:00 bl-cassoop-p11 kernel: [74707.606083]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.591612+00:00 bl-cassoop-p11 kernel: [74707.606084] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:48:55.603526+00:00 bl-cassoop-p11 kernel: [74707.612310] BUG: soft lockup - CPU#14 stuck for 22s! [java:42064]
2014-01-18T13:48:55.603535+00:00 bl-cassoop-p11 kernel: [74707.617808] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.603538+00:00 bl-cassoop-p11 kernel: [74707.617831] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.603540+00:00 bl-cassoop-p11 kernel: [74707.617832] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.603542+00:00 bl-cassoop-p11 kernel: [74707.617834] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:48:55.603544+00:00 bl-cassoop-p11 kernel: [74707.617836] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:48:55.603545+00:00 bl-cassoop-p11 kernel: [74707.617840] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:48:55.603547+00:00 bl-cassoop-p11 kernel: [74707.617841] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa986d0
2014-01-18T13:48:55.603548+00:00 bl-cassoop-p11 kernel: [74707.617843] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.603550+00:00 bl-cassoop-p11 kernel: [74707.617845] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:48:55.603551+00:00 bl-cassoop-p11 kernel: [74707.617846] R10: 0000000000000004 R11: 0000000000000058 R12: ffffffff81049a3e
2014-01-18T13:48:55.603552+00:00 bl-cassoop-p11 kernel: [74707.617848] R13: ffff88195e365078 R14: 0000000000000292 R15: ffff88195e3650a8
2014-01-18T13:48:55.603554+00:00 bl-cassoop-p11 kernel: [74707.617850] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.603555+00:00 bl-cassoop-p11 kernel: [74707.617852] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.603557+00:00 bl-cassoop-p11 kernel: [74707.617854] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:48:55.603558+00:00 bl-cassoop-p11 kernel: [74707.617855] Stack:
2014-01-18T13:48:55.603559+00:00 bl-cassoop-p11 kernel: [74707.617856]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:48:55.603561+00:00 bl-cassoop-p11 kernel: [74707.617862]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:48:55.603562+00:00 bl-cassoop-p11 kernel: [74707.617866]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:48:55.603563+00:00 bl-cassoop-p11 kernel: [74707.617871] Call Trace:
2014-01-18T13:48:55.603565+00:00 bl-cassoop-p11 kernel: [74707.617875]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.603566+00:00 bl-cassoop-p11 kernel: [74707.617879]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.603568+00:00 bl-cassoop-p11 kernel: [74707.617883]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:48:55.603569+00:00 bl-cassoop-p11 kernel: [74707.617886]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:48:55.603571+00:00 bl-cassoop-p11 kernel: [74707.617890]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:48:55.603573+00:00 bl-cassoop-p11 kernel: [74707.617893]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:48:55.603574+00:00 bl-cassoop-p11 kernel: [74707.617897]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:48:55.603575+00:00 bl-cassoop-p11 kernel: [74707.617901]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:48:55.603577+00:00 bl-cassoop-p11 kernel: [74707.617905]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:48:55.603578+00:00 bl-cassoop-p11 kernel: [74707.617909]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:48:55.603580+00:00 bl-cassoop-p11 kernel: [74707.617913]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:48:55.603581+00:00 bl-cassoop-p11 kernel: [74707.617916]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:48:55.603583+00:00 bl-cassoop-p11 kernel: [74707.617920]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:48:55.603584+00:00 bl-cassoop-p11 kernel: [74707.617925]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:48:55.603586+00:00 bl-cassoop-p11 kernel: [74707.617928]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:48:55.603588+00:00 bl-cassoop-p11 kernel: [74707.617932]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:48:55.603589+00:00 bl-cassoop-p11 kernel: [74707.617937]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:48:55.603591+00:00 bl-cassoop-p11 kernel: [74707.617941]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:48:55.603593+00:00 bl-cassoop-p11 kernel: [74707.617944]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:48:55.603594+00:00 bl-cassoop-p11 kernel: [74707.617948]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:48:55.603596+00:00 bl-cassoop-p11 kernel: [74707.617952]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:48:55.603597+00:00 bl-cassoop-p11 kernel: [74707.617956]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:48:55.603599+00:00 bl-cassoop-p11 kernel: [74707.617959]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:48:55.603601+00:00 bl-cassoop-p11 kernel: [74707.617964]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:48:55.603602+00:00 bl-cassoop-p11 kernel: [74707.617969]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:48:55.603604+00:00 bl-cassoop-p11 kernel: [74707.617974]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:48:55.603605+00:00 bl-cassoop-p11 kernel: [74707.617977]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:48:55.603607+00:00 bl-cassoop-p11 kernel: [74707.617982]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:48:55.603608+00:00 bl-cassoop-p11 kernel: [74707.617985]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:48:55.603610+00:00 bl-cassoop-p11 kernel: [74707.617988]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:48:55.603611+00:00 bl-cassoop-p11 kernel: [74707.617991]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:48:55.603613+00:00 bl-cassoop-p11 kernel: [74707.617995]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:48:55.603629+00:00 bl-cassoop-p11 kernel: [74707.617998]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:48:55.603630+00:00 bl-cassoop-p11 kernel: [74707.618002]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:48:55.603632+00:00 bl-cassoop-p11 kernel: [74707.618006]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:48:55.603646+00:00 bl-cassoop-p11 kernel: [74707.618009]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:48:55.603648+00:00 bl-cassoop-p11 kernel: [74707.618013]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:48:55.603649+00:00 bl-cassoop-p11 kernel: [74707.618016]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:48:55.603651+00:00 bl-cassoop-p11 kernel: [74707.618019]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:48:55.603652+00:00 bl-cassoop-p11 kernel: [74707.618023]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:48:55.603654+00:00 bl-cassoop-p11 kernel: [74707.618026]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:48:55.603655+00:00 bl-cassoop-p11 kernel: [74707.618029]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:48:55.603657+00:00 bl-cassoop-p11 kernel: [74707.618033]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:48:55.603658+00:00 bl-cassoop-p11 kernel: [74707.618037]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:48:55.603660+00:00 bl-cassoop-p11 kernel: [74707.618040]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:48:55.603661+00:00 bl-cassoop-p11 kernel: [74707.618044]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:48:55.603663+00:00 bl-cassoop-p11 kernel: [74707.618047]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:48:55.603664+00:00 bl-cassoop-p11 kernel: [74707.618051]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:48:55.603667+00:00 bl-cassoop-p11 kernel: [74707.618052] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:48:55.627358+00:00 bl-cassoop-p11 kernel: [74707.636324] BUG: soft lockup - CPU#16 stuck for 22s! [kworker/u97:9:42017]
2014-01-18T13:48:55.627366+00:00 bl-cassoop-p11 kernel: [74707.641711] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.627383+00:00 bl-cassoop-p11 kernel: [74707.641733] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.627385+00:00 bl-cassoop-p11 kernel: [74707.641735] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.627386+00:00 bl-cassoop-p11 kernel: [74707.641740] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:48:55.627388+00:00 bl-cassoop-p11 kernel: [74707.641743] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:48:55.627390+00:00 bl-cassoop-p11 kernel: [74707.641744] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:48:55.627391+00:00 bl-cassoop-p11 kernel: [74707.641748] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:48:55.627393+00:00 bl-cassoop-p11 kernel: [74707.641749] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa98720
2014-01-18T13:48:55.627394+00:00 bl-cassoop-p11 kernel: [74707.641751] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.627395+00:00 bl-cassoop-p11 kernel: [74707.641753] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:48:55.627397+00:00 bl-cassoop-p11 kernel: [74707.641754] R10: 0000000000000004 R11: 0000000000000210 R12: ffffffff81254cb7
2014-01-18T13:48:55.627398+00:00 bl-cassoop-p11 kernel: [74707.641756] R13: ffff880075d0b5c8 R14: 0000000000000292 R15: ffff880075d0b5d8
2014-01-18T13:48:55.627400+00:00 bl-cassoop-p11 kernel: [74707.641758] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.627401+00:00 bl-cassoop-p11 kernel: [74707.641760] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.627402+00:00 bl-cassoop-p11 kernel: [74707.641762] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:48:55.627403+00:00 bl-cassoop-p11 kernel: [74707.641763] Stack:
2014-01-18T13:48:55.627405+00:00 bl-cassoop-p11 kernel: [74707.641765]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:48:55.627406+00:00 bl-cassoop-p11 kernel: [74707.641770]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:48:55.627408+00:00 bl-cassoop-p11 kernel: [74707.641774]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:48:55.627409+00:00 bl-cassoop-p11 kernel: [74707.641779] Call Trace:
2014-01-18T13:48:55.627410+00:00 bl-cassoop-p11 kernel: [74707.641783]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.627412+00:00 bl-cassoop-p11 kernel: [74707.641787]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.627413+00:00 bl-cassoop-p11 kernel: [74707.641790]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:48:55.627415+00:00 bl-cassoop-p11 kernel: [74707.641794]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:48:55.627416+00:00 bl-cassoop-p11 kernel: [74707.641799]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:48:55.627418+00:00 bl-cassoop-p11 kernel: [74707.641802]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:48:55.627419+00:00 bl-cassoop-p11 kernel: [74707.641805]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:48:55.627421+00:00 bl-cassoop-p11 kernel: [74707.641809]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:48:55.627422+00:00 bl-cassoop-p11 kernel: [74707.641812]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:48:55.627424+00:00 bl-cassoop-p11 kernel: [74707.641816]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:48:55.627426+00:00 bl-cassoop-p11 kernel: [74707.641820]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:48:55.627427+00:00 bl-cassoop-p11 kernel: [74707.641826]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:48:55.627429+00:00 bl-cassoop-p11 kernel: [74707.641831]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:48:55.627431+00:00 bl-cassoop-p11 kernel: [74707.641835]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:48:55.627432+00:00 bl-cassoop-p11 kernel: [74707.641838]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:48:55.627434+00:00 bl-cassoop-p11 kernel: [74707.641842]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:48:55.627435+00:00 bl-cassoop-p11 kernel: [74707.641845]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:48:55.627437+00:00 bl-cassoop-p11 kernel: [74707.641849]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:48:55.627438+00:00 bl-cassoop-p11 kernel: [74707.641852]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:48:55.627440+00:00 bl-cassoop-p11 kernel: [74707.641855]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:48:55.627441+00:00 bl-cassoop-p11 kernel: [74707.641859]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:48:55.627443+00:00 bl-cassoop-p11 kernel: [74707.641863]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:48:55.627444+00:00 bl-cassoop-p11 kernel: [74707.641866]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:48:55.627446+00:00 bl-cassoop-p11 kernel: [74707.641871]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:48:55.627448+00:00 bl-cassoop-p11 kernel: [74707.641874]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:48:55.627449+00:00 bl-cassoop-p11 kernel: [74707.641877]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:48:55.627451+00:00 bl-cassoop-p11 kernel: [74707.641881]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:48:55.627452+00:00 bl-cassoop-p11 kernel: [74707.641885]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.627454+00:00 bl-cassoop-p11 kernel: [74707.641889]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:48:55.627455+00:00 bl-cassoop-p11 kernel: [74707.641892]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.627460+00:00 bl-cassoop-p11 kernel: [74707.641893] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:48:55.639126+00:00 bl-cassoop-p11 kernel: [74707.648323] BUG: soft lockup - CPU#17 stuck for 22s! [java:42521]
2014-01-18T13:48:55.639133+00:00 bl-cassoop-p11 kernel: [74707.653568] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.639135+00:00 bl-cassoop-p11 kernel: [74707.653593] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.639137+00:00 bl-cassoop-p11 kernel: [74707.653595] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.639138+00:00 bl-cassoop-p11 kernel: [74707.653597] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:48:55.639140+00:00 bl-cassoop-p11 kernel: [74707.653598] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:48:55.639142+00:00 bl-cassoop-p11 kernel: [74707.653601] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:48:55.639143+00:00 bl-cassoop-p11 kernel: [74707.653602] RAX: 0000000000000006 RBX: ffffffff81d03da0 RCX: ffff88181fa78748
2014-01-18T13:48:55.639145+00:00 bl-cassoop-p11 kernel: [74707.653603] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.639146+00:00 bl-cassoop-p11 kernel: [74707.653604] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:48:55.639147+00:00 bl-cassoop-p11 kernel: [74707.653605] R10: 0000000000000004 R11: 0000000000017960 R12: ffff880cff722548
2014-01-18T13:48:55.639149+00:00 bl-cassoop-p11 kernel: [74707.653606] R13: 0000000000000000 R14: 0000000000000292 R15: ffff882feccbf938
2014-01-18T13:48:55.639150+00:00 bl-cassoop-p11 kernel: [74707.653608] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.639152+00:00 bl-cassoop-p11 kernel: [74707.653609] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.639153+00:00 bl-cassoop-p11 kernel: [74707.653610] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:48:55.639154+00:00 bl-cassoop-p11 kernel: [74707.653611] Stack:
2014-01-18T13:48:55.639155+00:00 bl-cassoop-p11 kernel: [74707.653612]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:48:55.639157+00:00 bl-cassoop-p11 kernel: [74707.653616]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:48:55.639158+00:00 bl-cassoop-p11 kernel: [74707.653619]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:48:55.639159+00:00 bl-cassoop-p11 kernel: [74707.653622] Call Trace:
2014-01-18T13:48:55.639161+00:00 bl-cassoop-p11 kernel: [74707.653625]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.639163+00:00 bl-cassoop-p11 kernel: [74707.653628]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.639164+00:00 bl-cassoop-p11 kernel: [74707.653630]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:48:55.639166+00:00 bl-cassoop-p11 kernel: [74707.653633]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:48:55.639167+00:00 bl-cassoop-p11 kernel: [74707.653636]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:48:55.639169+00:00 bl-cassoop-p11 kernel: [74707.653638]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:48:55.639170+00:00 bl-cassoop-p11 kernel: [74707.653641]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:48:55.639172+00:00 bl-cassoop-p11 kernel: [74707.653643]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:48:55.639173+00:00 bl-cassoop-p11 kernel: [74707.653645]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:48:55.639175+00:00 bl-cassoop-p11 kernel: [74707.653648]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:48:55.639176+00:00 bl-cassoop-p11 kernel: [74707.653650]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:48:55.639178+00:00 bl-cassoop-p11 kernel: [74707.653652]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:48:55.639180+00:00 bl-cassoop-p11 kernel: [74707.653654]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:48:55.639181+00:00 bl-cassoop-p11 kernel: [74707.653659]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:48:55.639183+00:00 bl-cassoop-p11 kernel: [74707.653661]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:48:55.639184+00:00 bl-cassoop-p11 kernel: [74707.653664]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:48:55.639186+00:00 bl-cassoop-p11 kernel: [74707.653668]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:48:55.639187+00:00 bl-cassoop-p11 kernel: [74707.653670]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:48:55.639189+00:00 bl-cassoop-p11 kernel: [74707.653674]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:48:55.639190+00:00 bl-cassoop-p11 kernel: [74707.653677]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:48:55.639192+00:00 bl-cassoop-p11 kernel: [74707.653679]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:48:55.639193+00:00 bl-cassoop-p11 kernel: [74707.653681]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:48:55.639195+00:00 bl-cassoop-p11 kernel: [74707.653684]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:48:55.639197+00:00 bl-cassoop-p11 kernel: [74707.653685] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:48:55.651073+00:00 bl-cassoop-p11 kernel: [74707.660330] BUG: soft lockup - CPU#18 stuck for 22s! [kswapd0:283]
2014-01-18T13:48:55.651080+00:00 bl-cassoop-p11 kernel: [74707.665425] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.651095+00:00 bl-cassoop-p11 kernel: [74707.665463] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.651097+00:00 bl-cassoop-p11 kernel: [74707.665465] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.651098+00:00 bl-cassoop-p11 kernel: [74707.665467] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:48:55.651100+00:00 bl-cassoop-p11 kernel: [74707.665469] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:48:55.651102+00:00 bl-cassoop-p11 kernel: [74707.665473] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:48:55.651103+00:00 bl-cassoop-p11 kernel: [74707.665475] RAX: 0000000000000282 RBX: 0000000000000000 RCX: 0000000000000001
2014-01-18T13:48:55.651104+00:00 bl-cassoop-p11 kernel: [74707.665476] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:48:55.651106+00:00 bl-cassoop-p11 kernel: [74707.665478] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:48:55.651107+00:00 bl-cassoop-p11 kernel: [74707.665480] R10: 0000000000000001 R11: 0000000000000400 R12: ffffffff810a2fcc
2014-01-18T13:48:55.651109+00:00 bl-cassoop-p11 kernel: [74707.665482] R13: ffff882ff8361698 R14: ffff8817f2191200 R15: ffff882ff8361648
2014-01-18T13:48:55.651110+00:00 bl-cassoop-p11 kernel: [74707.665484] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.651111+00:00 bl-cassoop-p11 kernel: [74707.665486] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.651113+00:00 bl-cassoop-p11 kernel: [74707.665487] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:48:55.651114+00:00 bl-cassoop-p11 kernel: [74707.665489] Stack:
2014-01-18T13:48:55.651115+00:00 bl-cassoop-p11 kernel: [74707.665490]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:48:55.651117+00:00 bl-cassoop-p11 kernel: [74707.665495]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:48:55.651118+00:00 bl-cassoop-p11 kernel: [74707.665500]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:48:55.651119+00:00 bl-cassoop-p11 kernel: [74707.665505] Call Trace:
2014-01-18T13:48:55.651121+00:00 bl-cassoop-p11 kernel: [74707.665509]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.651122+00:00 bl-cassoop-p11 kernel: [74707.665512]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:48:55.651124+00:00 bl-cassoop-p11 kernel: [74707.665516]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.651125+00:00 bl-cassoop-p11 kernel: [74707.665519]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.651127+00:00 bl-cassoop-p11 kernel: [74707.665522]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:48:55.651129+00:00 bl-cassoop-p11 kernel: [74707.665526]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.651130+00:00 bl-cassoop-p11 kernel: [74707.665530]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.651132+00:00 bl-cassoop-p11 kernel: [74707.665533]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:48:55.651133+00:00 bl-cassoop-p11 kernel: [74707.665537]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:48:55.651135+00:00 bl-cassoop-p11 kernel: [74707.665542]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:48:55.651136+00:00 bl-cassoop-p11 kernel: [74707.665545]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:48:55.651138+00:00 bl-cassoop-p11 kernel: [74707.665549]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:48:55.651139+00:00 bl-cassoop-p11 kernel: [74707.665552]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:48:55.651141+00:00 bl-cassoop-p11 kernel: [74707.665556]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:48:55.651143+00:00 bl-cassoop-p11 kernel: [74707.665560]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:48:55.651144+00:00 bl-cassoop-p11 kernel: [74707.665563]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:48:55.651146+00:00 bl-cassoop-p11 kernel: [74707.665567]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:48:55.651147+00:00 bl-cassoop-p11 kernel: [74707.665571]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:48:55.651149+00:00 bl-cassoop-p11 kernel: [74707.665575]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:48:55.651151+00:00 bl-cassoop-p11 kernel: [74707.665579]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:48:55.651152+00:00 bl-cassoop-p11 kernel: [74707.665582]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:48:55.651154+00:00 bl-cassoop-p11 kernel: [74707.665586]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:48:55.651155+00:00 bl-cassoop-p11 kernel: [74707.665589]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:48:55.651157+00:00 bl-cassoop-p11 kernel: [74707.665593]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:48:55.651158+00:00 bl-cassoop-p11 kernel: [74707.665596]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:48:55.651160+00:00 bl-cassoop-p11 kernel: [74707.665600]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:48:55.651161+00:00 bl-cassoop-p11 kernel: [74707.665604]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:48:55.651163+00:00 bl-cassoop-p11 kernel: [74707.665607]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.651164+00:00 bl-cassoop-p11 kernel: [74707.665611]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:48:55.651166+00:00 bl-cassoop-p11 kernel: [74707.665614]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:48:55.651170+00:00 bl-cassoop-p11 kernel: [74707.665616] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:48:55.674758+00:00 bl-cassoop-p11 kernel: [74707.684338] BUG: soft lockup - CPU#20 stuck for 22s! [jsvc:42019]
2014-01-18T13:48:55.674765+00:00 bl-cassoop-p11 kernel: [74707.689262] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.674767+00:00 bl-cassoop-p11 kernel: [74707.689283] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.674769+00:00 bl-cassoop-p11 kernel: [74707.689284] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.674770+00:00 bl-cassoop-p11 kernel: [74707.689286] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:48:55.674773+00:00 bl-cassoop-p11 kernel: [74707.689287] RIP: 0010:[<ffffffff810cb9a4>]  [<ffffffff810cb9a4>] smp_call_function_many+0x274/0x2d0
2014-01-18T13:48:55.674774+00:00 bl-cassoop-p11 kernel: [74707.689290] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:48:55.674775+00:00 bl-cassoop-p11 kernel: [74707.689291] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa387c0
2014-01-18T13:48:55.674777+00:00 bl-cassoop-p11 kernel: [74707.689292] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:48:55.674778+00:00 bl-cassoop-p11 kernel: [74707.689293] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:48:55.674780+00:00 bl-cassoop-p11 kernel: [74707.689294] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:48:55.674781+00:00 bl-cassoop-p11 kernel: [74707.689295] R13: ffff8801111b3c48 R14: 0000000000000292 R15: ffff8801111b3c78
2014-01-18T13:48:55.674783+00:00 bl-cassoop-p11 kernel: [74707.689297] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.674784+00:00 bl-cassoop-p11 kernel: [74707.689298] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.674785+00:00 bl-cassoop-p11 kernel: [74707.689299] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:48:55.674786+00:00 bl-cassoop-p11 kernel: [74707.689300] Stack:
2014-01-18T13:48:55.674788+00:00 bl-cassoop-p11 kernel: [74707.689301]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:48:55.674789+00:00 bl-cassoop-p11 kernel: [74707.689305]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:48:55.674791+00:00 bl-cassoop-p11 kernel: [74707.689308]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:48:55.674792+00:00 bl-cassoop-p11 kernel: [74707.689311] Call Trace:
2014-01-18T13:48:55.674793+00:00 bl-cassoop-p11 kernel: [74707.689315]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.674795+00:00 bl-cassoop-p11 kernel: [74707.689317]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:48:55.674797+00:00 bl-cassoop-p11 kernel: [74707.689320]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:48:55.674798+00:00 bl-cassoop-p11 kernel: [74707.689323]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:48:55.674800+00:00 bl-cassoop-p11 kernel: [74707.689325]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:48:55.674801+00:00 bl-cassoop-p11 kernel: [74707.689328]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:48:55.674803+00:00 bl-cassoop-p11 kernel: [74707.689330]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:48:55.674804+00:00 bl-cassoop-p11 kernel: [74707.689332]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:48:55.674806+00:00 bl-cassoop-p11 kernel: [74707.689336]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:48:55.674808+00:00 bl-cassoop-p11 kernel: [74707.689336] Code: 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 74 c4 <eb> f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff 48 83
2014-01-18T13:48:55.758680+00:00 bl-cassoop-p11 kernel: [74707.768362] BUG: soft lockup - CPU#27 stuck for 23s! [java:42434]
2014-01-18T13:48:55.758688+00:00 bl-cassoop-p11 kernel: [74707.773123] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:48:55.758704+00:00 bl-cassoop-p11 kernel: [74707.773138] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:48:55.758706+00:00 bl-cassoop-p11 kernel: [74707.773140] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:48:55.758708+00:00 bl-cassoop-p11 kernel: [74707.773141] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:48:55.758710+00:00 bl-cassoop-p11 kernel: [74707.773142] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:48:55.758711+00:00 bl-cassoop-p11 kernel: [74707.773145] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:48:55.758713+00:00 bl-cassoop-p11 kernel: [74707.773147] RAX: 0000000000000030 RBX: ffffffff810a3789 RCX: 0000000000000010
2014-01-18T13:48:55.758720+00:00 bl-cassoop-p11 kernel: [74707.773148] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:48:55.758721+00:00 bl-cassoop-p11 kernel: [74707.773149] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:48:55.758723+00:00 bl-cassoop-p11 kernel: [74707.773150] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:48:55.758724+00:00 bl-cassoop-p11 kernel: [74707.773151] R13: 000000000000000e R14: ffffffff810a108e R15: ffff881887d733b8
2014-01-18T13:48:55.758726+00:00 bl-cassoop-p11 kernel: [74707.773152] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:48:55.758727+00:00 bl-cassoop-p11 kernel: [74707.773153] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:48:55.758728+00:00 bl-cassoop-p11 kernel: [74707.773155] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:48:55.758729+00:00 bl-cassoop-p11 kernel: [74707.773155] Stack:
2014-01-18T13:48:55.758731+00:00 bl-cassoop-p11 kernel: [74707.773156]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:48:55.758732+00:00 bl-cassoop-p11 kernel: [74707.773160]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:48:55.758734+00:00 bl-cassoop-p11 kernel: [74707.773163]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:48:55.758735+00:00 bl-cassoop-p11 kernel: [74707.773166] Call Trace:
2014-01-18T13:48:55.758736+00:00 bl-cassoop-p11 kernel: [74707.773169]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.758738+00:00 bl-cassoop-p11 kernel: [74707.773171]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:48:55.758739+00:00 bl-cassoop-p11 kernel: [74707.773174]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.758741+00:00 bl-cassoop-p11 kernel: [74707.773176]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:48:55.758742+00:00 bl-cassoop-p11 kernel: [74707.773178]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:48:55.758744+00:00 bl-cassoop-p11 kernel: [74707.773181]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:48:55.758746+00:00 bl-cassoop-p11 kernel: [74707.773183]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:48:55.758747+00:00 bl-cassoop-p11 kernel: [74707.773186]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:48:55.758749+00:00 bl-cassoop-p11 kernel: [74707.773189]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:48:55.758750+00:00 bl-cassoop-p11 kernel: [74707.773192]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:48:55.758752+00:00 bl-cassoop-p11 kernel: [74707.773195]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:48:55.758753+00:00 bl-cassoop-p11 kernel: [74707.773197]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:48:55.758755+00:00 bl-cassoop-p11 kernel: [74707.773200]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:48:55.758757+00:00 bl-cassoop-p11 kernel: [74707.773202]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:48:55.758758+00:00 bl-cassoop-p11 kernel: [74707.773205]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:48:55.758760+00:00 bl-cassoop-p11 kernel: [74707.773208]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:48:55.758761+00:00 bl-cassoop-p11 kernel: [74707.773211]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:48:55.758763+00:00 bl-cassoop-p11 kernel: [74707.773213]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:48:55.758764+00:00 bl-cassoop-p11 kernel: [74707.773216]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:48:55.758766+00:00 bl-cassoop-p11 kernel: [74707.773219]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:48:55.758767+00:00 bl-cassoop-p11 kernel: [74707.773222]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:48:55.758769+00:00 bl-cassoop-p11 kernel: [74707.773225]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:48:55.758770+00:00 bl-cassoop-p11 kernel: [74707.773227]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:48:55.758772+00:00 bl-cassoop-p11 kernel: [74707.773231]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:48:55.758774+00:00 bl-cassoop-p11 kernel: [74707.773233]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:48:55.758775+00:00 bl-cassoop-p11 kernel: [74707.773236]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:48:55.758777+00:00 bl-cassoop-p11 kernel: [74707.773239]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:48:55.758779+00:00 bl-cassoop-p11 kernel: [74707.773241]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:48:55.758780+00:00 bl-cassoop-p11 kernel: [74707.773244]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:48:55.758782+00:00 bl-cassoop-p11 kernel: [74707.773246]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:48:55.758783+00:00 bl-cassoop-p11 kernel: [74707.773248]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:48:55.758785+00:00 bl-cassoop-p11 kernel: [74707.773251]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:48:55.758796+00:00 bl-cassoop-p11 kernel: [74707.773253]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:48:55.758809+00:00 bl-cassoop-p11 kernel: [74707.773255]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:48:55.758811+00:00 bl-cassoop-p11 kernel: [74707.773258]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:48:55.758812+00:00 bl-cassoop-p11 kernel: [74707.773261]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:48:55.758814+00:00 bl-cassoop-p11 kernel: [74707.773264]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:48:55.758816+00:00 bl-cassoop-p11 kernel: [74707.773266]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:48:55.758817+00:00 bl-cassoop-p11 kernel: [74707.773269]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:48:55.758819+00:00 bl-cassoop-p11 kernel: [74707.773271]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:48:55.758820+00:00 bl-cassoop-p11 kernel: [74707.773273]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:48:55.758822+00:00 bl-cassoop-p11 kernel: [74707.773274] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:19.794555+00:00 bl-cassoop-p11 kernel: [74720.544184] ------------[ cut here ]------------
2014-01-18T13:49:19.794568+00:00 bl-cassoop-p11 kernel: [74720.544190] WARNING: CPU: 6 PID: 5135 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:49:19.794571+00:00 bl-cassoop-p11 kernel: [74720.544192] Watchdog detected hard LOCKUP on cpu 6
2014-01-18T13:49:19.794574+00:00 bl-cassoop-p11 kernel: [74720.544193] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:19.794577+00:00 bl-cassoop-p11 kernel: [74720.544209] CPU: 6 PID: 5135 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:19.794578+00:00 bl-cassoop-p11 kernel: [74720.544210] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:19.794580+00:00 bl-cassoop-p11 kernel: [74720.544212]  00000000000000f5 ffff88181fa67ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:49:19.794581+00:00 bl-cassoop-p11 kernel: [74720.544216]  ffff88181fa67bf8 ffff88181fa67be8 ffffffff810653ac 0000000000000000
2014-01-18T13:49:19.794583+00:00 bl-cassoop-p11 kernel: [74720.544219]  ffff8817fa860000 0000000000000000 ffff88181fa67d20 0000000000000000
2014-01-18T13:49:19.794584+00:00 bl-cassoop-p11 kernel: [74720.544222] Call Trace:
2014-01-18T13:49:19.794586+00:00 bl-cassoop-p11 kernel: [74720.544223]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:49:19.794588+00:00 bl-cassoop-p11 kernel: [74720.544229]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:49:19.794590+00:00 bl-cassoop-p11 kernel: [74720.544231]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:49:19.794592+00:00 bl-cassoop-p11 kernel: [74720.544234]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:49:19.794593+00:00 bl-cassoop-p11 kernel: [74720.544237]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:49:19.794640+00:00 bl-cassoop-p11 kernel: [74720.544241]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:49:19.794642+00:00 bl-cassoop-p11 kernel: [74720.544244]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:49:19.794644+00:00 bl-cassoop-p11 kernel: [74720.544247]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:49:19.794646+00:00 bl-cassoop-p11 kernel: [74720.544250]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:49:19.794648+00:00 bl-cassoop-p11 kernel: [74720.544253]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:49:19.794649+00:00 bl-cassoop-p11 kernel: [74720.544257]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:49:19.794651+00:00 bl-cassoop-p11 kernel: [74720.544259]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:49:19.794653+00:00 bl-cassoop-p11 kernel: [74720.544262]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:49:19.794665+00:00 bl-cassoop-p11 kernel: [74720.544265]  [<ffffffff81748ea8>] default_do_nmi+0x58/0x240
2014-01-18T13:49:19.794667+00:00 bl-cassoop-p11 kernel: [74720.544267]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:49:19.794669+00:00 bl-cassoop-p11 kernel: [74720.544270]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:49:19.794671+00:00 bl-cassoop-p11 kernel: [74720.544274]  [<ffffffff813817cd>] ? __write_lock_failed+0xd/0x20
2014-01-18T13:49:19.794673+00:00 bl-cassoop-p11 kernel: [74720.544276]  [<ffffffff813817cd>] ? __write_lock_failed+0xd/0x20
2014-01-18T13:49:19.794674+00:00 bl-cassoop-p11 kernel: [74720.544278]  [<ffffffff813817cd>] ? __write_lock_failed+0xd/0x20
2014-01-18T13:49:19.794676+00:00 bl-cassoop-p11 kernel: [74720.544279]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:19.794678+00:00 bl-cassoop-p11 kernel: [74720.544283]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:19.794679+00:00 bl-cassoop-p11 kernel: [74720.544288]  [<ffffffff8167e2e0>] ? do_tcp_setsockopt.isra.29.part.30+0x60/0x720
2014-01-18T13:49:19.794681+00:00 bl-cassoop-p11 kernel: [74720.544291]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:19.794683+00:00 bl-cassoop-p11 kernel: [74720.544293]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:19.794685+00:00 bl-cassoop-p11 kernel: [74720.544295]  [<ffffffff811c4ae8>] ? setfl+0x118/0x170
2014-01-18T13:49:19.794687+00:00 bl-cassoop-p11 kernel: [74720.544297]  [<ffffffff811c5038>] ? do_fcntl+0x258/0x340
2014-01-18T13:49:19.794688+00:00 bl-cassoop-p11 kernel: [74720.544300]  [<ffffffff8167eab3>] ? tcp_setsockopt+0x33/0x40
2014-01-18T13:49:19.794690+00:00 bl-cassoop-p11 kernel: [74720.544302]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:19.794691+00:00 bl-cassoop-p11 kernel: [74720.544304]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:19.794693+00:00 bl-cassoop-p11 kernel: [74720.544307]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:19.794694+00:00 bl-cassoop-p11 kernel: [74720.544308] ---[ end trace 51e5206791572eff ]---
2014-01-18T13:49:19.794696+00:00 bl-cassoop-p11 kernel: [74730.272632] ------------[ cut here ]------------
2014-01-18T13:49:19.794698+00:00 bl-cassoop-p11 kernel: [74730.272638] WARNING: CPU: 8 PID: 42507 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:49:19.794700+00:00 bl-cassoop-p11 kernel: [74730.272639] Watchdog detected hard LOCKUP on cpu 8
2014-01-18T13:49:19.794703+00:00 bl-cassoop-p11 kernel: [74730.272640] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:19.794705+00:00 bl-cassoop-p11 kernel: [74730.272657] CPU: 8 PID: 42507 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:19.794707+00:00 bl-cassoop-p11 kernel: [74730.272658] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:19.794753+00:00 bl-cassoop-p11 kernel: [74730.272659]  00000000000000f5 ffff88181fa87ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:49:19.794755+00:00 bl-cassoop-p11 kernel: [74730.272663]  ffff88181fa87bf8 ffff88181fa87be8 ffffffff810653ac 0000000000000000
2014-01-18T13:49:19.794757+00:00 bl-cassoop-p11 kernel: [74730.272666]  ffff8817fa888000 0000000000000000 ffff88181fa87d20 0000000000000000
2014-01-18T13:49:19.794759+00:00 bl-cassoop-p11 kernel: [74730.272669] Call Trace:
2014-01-18T13:49:19.794760+00:00 bl-cassoop-p11 kernel: [74730.272670]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:49:19.794774+00:00 bl-cassoop-p11 kernel: [74730.272676]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:49:19.794776+00:00 bl-cassoop-p11 kernel: [74730.272678]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:49:19.794778+00:00 bl-cassoop-p11 kernel: [74730.272680]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:49:19.794780+00:00 bl-cassoop-p11 kernel: [74730.272683]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:49:19.794782+00:00 bl-cassoop-p11 kernel: [74730.272686]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:49:19.794783+00:00 bl-cassoop-p11 kernel: [74730.272688]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:49:19.794785+00:00 bl-cassoop-p11 kernel: [74730.272691]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:49:19.794786+00:00 bl-cassoop-p11 kernel: [74730.272694]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:49:19.794788+00:00 bl-cassoop-p11 kernel: [74730.272696]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:49:19.794790+00:00 bl-cassoop-p11 kernel: [74730.272699]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:49:19.794791+00:00 bl-cassoop-p11 kernel: [74730.272702]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:49:19.794793+00:00 bl-cassoop-p11 kernel: [74730.272704]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:49:19.794794+00:00 bl-cassoop-p11 kernel: [74730.272707]  [<ffffffff81748ea8>] default_do_nmi+0x58/0x240
2014-01-18T13:49:19.794796+00:00 bl-cassoop-p11 kernel: [74730.272709]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:49:19.794797+00:00 bl-cassoop-p11 kernel: [74730.272712]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:49:19.794799+00:00 bl-cassoop-p11 kernel: [74730.272715]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:19.794800+00:00 bl-cassoop-p11 kernel: [74730.272717]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:19.794802+00:00 bl-cassoop-p11 kernel: [74730.272719]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:19.794804+00:00 bl-cassoop-p11 kernel: [74730.272720]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:19.794806+00:00 bl-cassoop-p11 kernel: [74730.272725]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:19.794807+00:00 bl-cassoop-p11 kernel: [74730.272727]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:19.794808+00:00 bl-cassoop-p11 kernel: [74730.272729]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:19.794810+00:00 bl-cassoop-p11 kernel: [74730.272732]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:19.794812+00:00 bl-cassoop-p11 kernel: [74730.272734]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:19.794813+00:00 bl-cassoop-p11 kernel: [74730.272736] ---[ end trace 51e5206791572f00 ]---
2014-01-18T13:49:19.794815+00:00 bl-cassoop-p11 kernel: [74731.813143] BUG: soft lockup - CPU#30 stuck for 22s! [java:36244]
2014-01-18T13:49:19.794819+00:00 bl-cassoop-p11 kernel: [74731.817750] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:19.794821+00:00 bl-cassoop-p11 kernel: [74731.817772] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:19.794822+00:00 bl-cassoop-p11 kernel: [74731.817774] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:19.794824+00:00 bl-cassoop-p11 kernel: [74731.817776] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:49:19.794826+00:00 bl-cassoop-p11 kernel: [74731.817778] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:19.794827+00:00 bl-cassoop-p11 kernel: [74731.817781] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:49:19.794829+00:00 bl-cassoop-p11 kernel: [74731.817783] RAX: 0000000000000286 RBX: ffff88187fffcb08 RCX: 0000000000000001
2014-01-18T13:49:19.794831+00:00 bl-cassoop-p11 kernel: [74731.817785] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:19.794832+00:00 bl-cassoop-p11 kernel: [74731.817786] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:49:19.794833+00:00 bl-cassoop-p11 kernel: [74731.817788] R10: 00000000000035de R11: 0000000000000001 R12: 0000000000000000
2014-01-18T13:49:19.794835+00:00 bl-cassoop-p11 kernel: [74731.817790] R13: ffff88187fffcb00 R14: ffff88187fffcb08 R15: ffffffff00000000
2014-01-18T13:49:19.794836+00:00 bl-cassoop-p11 kernel: [74731.817792] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:49:19.794838+00:00 bl-cassoop-p11 kernel: [74731.817794] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:19.794840+00:00 bl-cassoop-p11 kernel: [74731.817795] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:19.794841+00:00 bl-cassoop-p11 kernel: [74731.817797] Stack:
2014-01-18T13:49:19.794842+00:00 bl-cassoop-p11 kernel: [74731.817798]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:49:19.794844+00:00 bl-cassoop-p11 kernel: [74731.817804]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:49:19.794845+00:00 bl-cassoop-p11 kernel: [74731.817808]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:49:19.794846+00:00 bl-cassoop-p11 kernel: [74731.817813] Call Trace:
2014-01-18T13:49:19.794848+00:00 bl-cassoop-p11 kernel: [74731.817817]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:19.794871+00:00 bl-cassoop-p11 kernel: [74731.817820]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:19.794874+00:00 bl-cassoop-p11 kernel: [74731.817823]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:19.794875+00:00 bl-cassoop-p11 kernel: [74731.817826]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:19.794877+00:00 bl-cassoop-p11 kernel: [74731.817829]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:19.794879+00:00 bl-cassoop-p11 kernel: [74731.817832]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:49:19.794880+00:00 bl-cassoop-p11 kernel: [74731.817836]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:19.794882+00:00 bl-cassoop-p11 kernel: [74731.817840]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:19.794884+00:00 bl-cassoop-p11 kernel: [74731.817844]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:49:19.794886+00:00 bl-cassoop-p11 kernel: [74731.817846]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:49:19.794888+00:00 bl-cassoop-p11 kernel: [74731.817850]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:49:19.794889+00:00 bl-cassoop-p11 kernel: [74731.817853]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:49:19.794891+00:00 bl-cassoop-p11 kernel: [74731.817856]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:49:19.794892+00:00 bl-cassoop-p11 kernel: [74731.817859]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:49:19.794894+00:00 bl-cassoop-p11 kernel: [74731.817862]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:49:19.794895+00:00 bl-cassoop-p11 kernel: [74731.817866]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:49:19.794897+00:00 bl-cassoop-p11 kernel: [74731.817869]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:49:19.794899+00:00 bl-cassoop-p11 kernel: [74731.817872]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:49:19.794901+00:00 bl-cassoop-p11 kernel: [74731.817875]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:49:19.794902+00:00 bl-cassoop-p11 kernel: [74731.817879]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:49:19.794904+00:00 bl-cassoop-p11 kernel: [74731.817882]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:49:19.794905+00:00 bl-cassoop-p11 kernel: [74731.817885]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:19.794907+00:00 bl-cassoop-p11 kernel: [74731.817889]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:19.794908+00:00 bl-cassoop-p11 kernel: [74731.817892]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:19.794910+00:00 bl-cassoop-p11 kernel: [74731.817896]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:19.794912+00:00 bl-cassoop-p11 kernel: [74731.817899]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:49:19.794914+00:00 bl-cassoop-p11 kernel: [74731.817903]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:19.794915+00:00 bl-cassoop-p11 kernel: [74731.817906]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:49:19.794917+00:00 bl-cassoop-p11 kernel: [74731.817908] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:23.442391+00:00 bl-cassoop-p11 kernel: [74735.462476] BUG: soft lockup - CPU#0 stuck for 22s! [java:42506]
2014-01-18T13:49:23.442402+00:00 bl-cassoop-p11 kernel: [74735.466987] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.442405+00:00 bl-cassoop-p11 kernel: [74735.467017] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.442407+00:00 bl-cassoop-p11 kernel: [74735.467019] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.442408+00:00 bl-cassoop-p11 kernel: [74735.467021] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:49:23.442411+00:00 bl-cassoop-p11 kernel: [74735.467023] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:23.442412+00:00 bl-cassoop-p11 kernel: [74735.467027] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:49:23.442413+00:00 bl-cassoop-p11 kernel: [74735.467029] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa97868
2014-01-18T13:49:23.442415+00:00 bl-cassoop-p11 kernel: [74735.467031] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.442416+00:00 bl-cassoop-p11 kernel: [74735.467032] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:49:23.442418+00:00 bl-cassoop-p11 kernel: [74735.467034] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:49:23.442420+00:00 bl-cassoop-p11 kernel: [74735.467036] R13: ffff880045ccfc48 R14: 0000000000000292 R15: ffff880045ccfc78
2014-01-18T13:49:23.442421+00:00 bl-cassoop-p11 kernel: [74735.467038] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.442423+00:00 bl-cassoop-p11 kernel: [74735.467040] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.442424+00:00 bl-cassoop-p11 kernel: [74735.467042] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:49:23.442425+00:00 bl-cassoop-p11 kernel: [74735.467043] Stack:
2014-01-18T13:49:23.442462+00:00 bl-cassoop-p11 kernel: [74735.467044]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:49:23.442464+00:00 bl-cassoop-p11 kernel: [74735.467051]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:49:23.442465+00:00 bl-cassoop-p11 kernel: [74735.467055]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:49:23.442466+00:00 bl-cassoop-p11 kernel: [74735.467060] Call Trace:
2014-01-18T13:49:23.442468+00:00 bl-cassoop-p11 kernel: [74735.467064]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.442470+00:00 bl-cassoop-p11 kernel: [74735.467068]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:23.442471+00:00 bl-cassoop-p11 kernel: [74735.467072]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:23.442484+00:00 bl-cassoop-p11 kernel: [74735.467075]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:23.442486+00:00 bl-cassoop-p11 kernel: [74735.467078]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:23.442488+00:00 bl-cassoop-p11 kernel: [74735.467082]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:23.442489+00:00 bl-cassoop-p11 kernel: [74735.467085]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:23.442491+00:00 bl-cassoop-p11 kernel: [74735.467089]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:23.442493+00:00 bl-cassoop-p11 kernel: [74735.467092]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:23.442495+00:00 bl-cassoop-p11 kernel: [74735.467094] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:23.554294+00:00 bl-cassoop-p11 kernel: [74735.574517] BUG: soft lockup - CPU#10 stuck for 22s! [java:42435]
2014-01-18T13:49:23.554302+00:00 bl-cassoop-p11 kernel: [74735.578924] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.554304+00:00 bl-cassoop-p11 kernel: [74735.578955] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.554305+00:00 bl-cassoop-p11 kernel: [74735.578956] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.554307+00:00 bl-cassoop-p11 kernel: [74735.578958] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:49:23.554309+00:00 bl-cassoop-p11 kernel: [74735.578959] RIP: 0010:[<ffffffff81731068>]  [<ffffffff81731068>] oom_kill_process.part.6+0xae/0x2cf
2014-01-18T13:49:23.554310+00:00 bl-cassoop-p11 kernel: [74735.578963] RSP: 0000:ffff8819e9a81a28  EFLAGS: 00000206
2014-01-18T13:49:23.554312+00:00 bl-cassoop-p11 kernel: [74735.578964] RAX: ffff8818979f6150 RBX: 00000000000000d0 RCX: 0000000000000006
2014-01-18T13:49:23.554313+00:00 bl-cassoop-p11 kernel: [74735.578965] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffffffff81c06040
2014-01-18T13:49:23.554315+00:00 bl-cassoop-p11 kernel: [74735.578966] RBP: ffff8819e9a81a88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:23.554316+00:00 bl-cassoop-p11 kernel: [74735.578967] R10: 0000000000000e32 R11: 0000000000005000 R12: ffff882ff044e3f0
2014-01-18T13:49:23.554317+00:00 bl-cassoop-p11 kernel: [74735.578968] R13: 0000000000000005 R14: 0000000000000246 R15: 0000000028e74000
2014-01-18T13:49:23.554319+00:00 bl-cassoop-p11 kernel: [74735.578970] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.554320+00:00 bl-cassoop-p11 kernel: [74735.578971] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.554322+00:00 bl-cassoop-p11 kernel: [74735.578972] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:23.554323+00:00 bl-cassoop-p11 kernel: [74735.578973] Stack:
2014-01-18T13:49:23.554324+00:00 bl-cassoop-p11 kernel: [74735.578974]  ffff8819e9a81a38 ffff882ff044ddc0 000000000114eccd ffffc90028e74000
2014-01-18T13:49:23.554326+00:00 bl-cassoop-p11 kernel: [74735.578978]  ffff8819e9a81a88 ffff8818979f6150 ffff882ff5c55dc0 000000000114eccd
2014-01-18T13:49:23.554359+00:00 bl-cassoop-p11 kernel: [74735.578981]  0000000000000000 ffffc90028e74000 ffff882ff0b86a00 ffff882ff044ddc0
2014-01-18T13:49:23.554360+00:00 bl-cassoop-p11 kernel: [74735.578984] Call Trace:
2014-01-18T13:49:23.554362+00:00 bl-cassoop-p11 kernel: [74735.578987]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:49:23.554364+00:00 bl-cassoop-p11 kernel: [74735.578989]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:49:23.554377+00:00 bl-cassoop-p11 kernel: [74735.578991]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:49:23.554379+00:00 bl-cassoop-p11 kernel: [74735.578994]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:49:23.554381+00:00 bl-cassoop-p11 kernel: [74735.578996]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:49:23.554382+00:00 bl-cassoop-p11 kernel: [74735.578999]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:49:23.554384+00:00 bl-cassoop-p11 kernel: [74735.579002]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:23.554385+00:00 bl-cassoop-p11 kernel: [74735.579004]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:23.554387+00:00 bl-cassoop-p11 kernel: [74735.579007]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:23.554389+00:00 bl-cassoop-p11 kernel: [74735.579010]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:23.554390+00:00 bl-cassoop-p11 kernel: [74735.579012]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:23.554392+00:00 bl-cassoop-p11 kernel: [74735.579014]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:23.554394+00:00 bl-cassoop-p11 kernel: [74735.579017]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:23.554395+00:00 bl-cassoop-p11 kernel: [74735.579019]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:23.554397+00:00 bl-cassoop-p11 kernel: [74735.579021]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:49:23.554399+00:00 bl-cassoop-p11 kernel: [74735.579023]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:23.554400+00:00 bl-cassoop-p11 kernel: [74735.579026]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:23.554402+00:00 bl-cassoop-p11 kernel: [74735.579028]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:23.554403+00:00 bl-cassoop-p11 kernel: [74735.579031]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:23.554405+00:00 bl-cassoop-p11 kernel: [74735.579033]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:23.554407+00:00 bl-cassoop-p11 kernel: [74735.579035]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:23.554408+00:00 bl-cassoop-p11 kernel: [74735.579038]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:23.554410+00:00 bl-cassoop-p11 kernel: [74735.579040]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:23.554412+00:00 bl-cassoop-p11 kernel: [74735.579041] Code: d1 e5 ff ff 48 89 df e8 e7 ff 91 ff 66 90 48 c7 c7 40 60 c0 81 4d 89 fe 45 31 ed e8 63 65 01 00 4c 89 7d a8 49 8b 9e 00 03 00 00 <4d> 8d a6 00 03 00 00 48 81 eb 10 03 00 00 eb 53 49 8b 87 a0 02
2014-01-18T13:49:23.578083+00:00 bl-cassoop-p11 kernel: [74735.598526] BUG: soft lockup - CPU#12 stuck for 22s! [jsvc:7242]
2014-01-18T13:49:23.578090+00:00 bl-cassoop-p11 kernel: [74735.602754] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.578092+00:00 bl-cassoop-p11 kernel: [74735.602771] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.578094+00:00 bl-cassoop-p11 kernel: [74735.602772] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.578095+00:00 bl-cassoop-p11 kernel: [74735.602774] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:49:23.578098+00:00 bl-cassoop-p11 kernel: [74735.602775] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:23.578099+00:00 bl-cassoop-p11 kernel: [74735.602778] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:49:23.578100+00:00 bl-cassoop-p11 kernel: [74735.602779] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa38680
2014-01-18T13:49:23.578102+00:00 bl-cassoop-p11 kernel: [74735.602780] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.578103+00:00 bl-cassoop-p11 kernel: [74735.602781] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:49:23.578105+00:00 bl-cassoop-p11 kernel: [74735.602782] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff8817f37ca000
2014-01-18T13:49:23.578106+00:00 bl-cassoop-p11 kernel: [74735.602783] R13: 0000000000000001 R14: 0000000000000292 R15: ffff8817f8e45938
2014-01-18T13:49:23.578107+00:00 bl-cassoop-p11 kernel: [74735.602785] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.578109+00:00 bl-cassoop-p11 kernel: [74735.602786] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.578110+00:00 bl-cassoop-p11 kernel: [74735.602787] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:23.578111+00:00 bl-cassoop-p11 kernel: [74735.602788] Stack:
2014-01-18T13:49:23.578113+00:00 bl-cassoop-p11 kernel: [74735.602789]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:49:23.578114+00:00 bl-cassoop-p11 kernel: [74735.602793]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:49:23.578116+00:00 bl-cassoop-p11 kernel: [74735.602796]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:49:23.578118+00:00 bl-cassoop-p11 kernel: [74735.602799] Call Trace:
2014-01-18T13:49:23.578119+00:00 bl-cassoop-p11 kernel: [74735.602802]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.578121+00:00 bl-cassoop-p11 kernel: [74735.602805]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.578122+00:00 bl-cassoop-p11 kernel: [74735.602807]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:23.578124+00:00 bl-cassoop-p11 kernel: [74735.602809]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:23.578126+00:00 bl-cassoop-p11 kernel: [74735.602812]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:23.578138+00:00 bl-cassoop-p11 kernel: [74735.602814]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:23.578140+00:00 bl-cassoop-p11 kernel: [74735.602817]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:23.578142+00:00 bl-cassoop-p11 kernel: [74735.602819]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:23.578143+00:00 bl-cassoop-p11 kernel: [74735.602821]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:23.578145+00:00 bl-cassoop-p11 kernel: [74735.602823]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:23.578147+00:00 bl-cassoop-p11 kernel: [74735.602825]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:23.578148+00:00 bl-cassoop-p11 kernel: [74735.602827]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:23.578150+00:00 bl-cassoop-p11 kernel: [74735.602829]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:23.578151+00:00 bl-cassoop-p11 kernel: [74735.602832]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:49:23.578153+00:00 bl-cassoop-p11 kernel: [74735.602834]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:23.578155+00:00 bl-cassoop-p11 kernel: [74735.602836]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:23.578156+00:00 bl-cassoop-p11 kernel: [74735.602838]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:23.578158+00:00 bl-cassoop-p11 kernel: [74735.602841]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:23.578160+00:00 bl-cassoop-p11 kernel: [74735.602842] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:23.589982+00:00 bl-cassoop-p11 kernel: [74735.610540] BUG: soft lockup - CPU#13 stuck for 22s! [kswapd1:284]
2014-01-18T13:49:23.589991+00:00 bl-cassoop-p11 kernel: [74735.614633] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.589993+00:00 bl-cassoop-p11 kernel: [74735.614652] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.589995+00:00 bl-cassoop-p11 kernel: [74735.614654] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.589997+00:00 bl-cassoop-p11 kernel: [74735.614655] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:49:23.589999+00:00 bl-cassoop-p11 kernel: [74735.614656] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:23.590000+00:00 bl-cassoop-p11 kernel: [74735.614661] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:49:23.590001+00:00 bl-cassoop-p11 kernel: [74735.614662] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa386a8
2014-01-18T13:49:23.590003+00:00 bl-cassoop-p11 kernel: [74735.614663] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.590004+00:00 bl-cassoop-p11 kernel: [74735.614664] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:49:23.590006+00:00 bl-cassoop-p11 kernel: [74735.614665] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000009
2014-01-18T13:49:23.590007+00:00 bl-cassoop-p11 kernel: [74735.614666] R13: ffffffff8105ae90 R14: 0000000000000292 R15: ffff882ff8363748
2014-01-18T13:49:23.590009+00:00 bl-cassoop-p11 kernel: [74735.614668] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.590010+00:00 bl-cassoop-p11 kernel: [74735.614669] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.590011+00:00 bl-cassoop-p11 kernel: [74735.614670] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:23.590012+00:00 bl-cassoop-p11 kernel: [74735.614671] Stack:
2014-01-18T13:49:23.590014+00:00 bl-cassoop-p11 kernel: [74735.614672]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:49:23.590015+00:00 bl-cassoop-p11 kernel: [74735.614676]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:49:23.590017+00:00 bl-cassoop-p11 kernel: [74735.614679]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:49:23.590053+00:00 bl-cassoop-p11 kernel: [74735.614682] Call Trace:
2014-01-18T13:49:23.590056+00:00 bl-cassoop-p11 kernel: [74735.614688]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.590057+00:00 bl-cassoop-p11 kernel: [74735.614691]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.590059+00:00 bl-cassoop-p11 kernel: [74735.614694]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:23.590072+00:00 bl-cassoop-p11 kernel: [74735.614697]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:23.590073+00:00 bl-cassoop-p11 kernel: [74735.614699]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:23.590075+00:00 bl-cassoop-p11 kernel: [74735.614702]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:23.590077+00:00 bl-cassoop-p11 kernel: [74735.614705]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:23.590078+00:00 bl-cassoop-p11 kernel: [74735.614708]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:23.590080+00:00 bl-cassoop-p11 kernel: [74735.614711]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:23.590082+00:00 bl-cassoop-p11 kernel: [74735.614713]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:23.590083+00:00 bl-cassoop-p11 kernel: [74735.614716]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:23.590085+00:00 bl-cassoop-p11 kernel: [74735.614719]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:23.590087+00:00 bl-cassoop-p11 kernel: [74735.614721]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:23.590088+00:00 bl-cassoop-p11 kernel: [74735.614724]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:23.590090+00:00 bl-cassoop-p11 kernel: [74735.614726]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:23.590092+00:00 bl-cassoop-p11 kernel: [74735.614730]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:23.590093+00:00 bl-cassoop-p11 kernel: [74735.614732]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:23.590095+00:00 bl-cassoop-p11 kernel: [74735.614735]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:23.590096+00:00 bl-cassoop-p11 kernel: [74735.614738]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:23.590098+00:00 bl-cassoop-p11 kernel: [74735.614740]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.590099+00:00 bl-cassoop-p11 kernel: [74735.614745]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:23.590101+00:00 bl-cassoop-p11 kernel: [74735.614747]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.590103+00:00 bl-cassoop-p11 kernel: [74735.614748] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:23.601967+00:00 bl-cassoop-p11 kernel: [74735.622538] BUG: soft lockup - CPU#14 stuck for 22s! [java:42064]
2014-01-18T13:49:23.601976+00:00 bl-cassoop-p11 kernel: [74735.626485] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.601978+00:00 bl-cassoop-p11 kernel: [74735.626507] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.601980+00:00 bl-cassoop-p11 kernel: [74735.626509] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.601981+00:00 bl-cassoop-p11 kernel: [74735.626511] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:49:23.601984+00:00 bl-cassoop-p11 kernel: [74735.626512] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:23.601985+00:00 bl-cassoop-p11 kernel: [74735.626516] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:49:23.601986+00:00 bl-cassoop-p11 kernel: [74735.626517] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa986d0
2014-01-18T13:49:23.601988+00:00 bl-cassoop-p11 kernel: [74735.626519] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.601989+00:00 bl-cassoop-p11 kernel: [74735.626521] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:49:23.601991+00:00 bl-cassoop-p11 kernel: [74735.626522] R10: 0000000000000004 R11: 0000000000000058 R12: ffffffff81049a3e
2014-01-18T13:49:23.601992+00:00 bl-cassoop-p11 kernel: [74735.626524] R13: ffff88195e365078 R14: 0000000000000292 R15: ffff88195e3650a8
2014-01-18T13:49:23.601993+00:00 bl-cassoop-p11 kernel: [74735.626526] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.601995+00:00 bl-cassoop-p11 kernel: [74735.626528] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.601996+00:00 bl-cassoop-p11 kernel: [74735.626530] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:49:23.601997+00:00 bl-cassoop-p11 kernel: [74735.626531] Stack:
2014-01-18T13:49:23.601999+00:00 bl-cassoop-p11 kernel: [74735.626532]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:49:23.602000+00:00 bl-cassoop-p11 kernel: [74735.626538]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:49:23.602002+00:00 bl-cassoop-p11 kernel: [74735.626543]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:49:23.602033+00:00 bl-cassoop-p11 kernel: [74735.626547] Call Trace:
2014-01-18T13:49:23.602036+00:00 bl-cassoop-p11 kernel: [74735.626551]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.602038+00:00 bl-cassoop-p11 kernel: [74735.626555]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.602039+00:00 bl-cassoop-p11 kernel: [74735.626558]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:23.602041+00:00 bl-cassoop-p11 kernel: [74735.626562]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:23.602042+00:00 bl-cassoop-p11 kernel: [74735.626565]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:23.602056+00:00 bl-cassoop-p11 kernel: [74735.626569]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:23.602057+00:00 bl-cassoop-p11 kernel: [74735.626572]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:23.602059+00:00 bl-cassoop-p11 kernel: [74735.626576]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:23.602061+00:00 bl-cassoop-p11 kernel: [74735.626579]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:23.602062+00:00 bl-cassoop-p11 kernel: [74735.626583]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:23.602064+00:00 bl-cassoop-p11 kernel: [74735.626587]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:23.602065+00:00 bl-cassoop-p11 kernel: [74735.626590]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:23.602067+00:00 bl-cassoop-p11 kernel: [74735.626593]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:23.602068+00:00 bl-cassoop-p11 kernel: [74735.626597]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:23.602070+00:00 bl-cassoop-p11 kernel: [74735.626601]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:23.602072+00:00 bl-cassoop-p11 kernel: [74735.626604]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:23.602073+00:00 bl-cassoop-p11 kernel: [74735.626608]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:23.602075+00:00 bl-cassoop-p11 kernel: [74735.626612]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:23.602077+00:00 bl-cassoop-p11 kernel: [74735.626615]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:23.602078+00:00 bl-cassoop-p11 kernel: [74735.626619]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:23.602080+00:00 bl-cassoop-p11 kernel: [74735.626622]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:49:23.602082+00:00 bl-cassoop-p11 kernel: [74735.626626]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:49:23.602083+00:00 bl-cassoop-p11 kernel: [74735.626629]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:49:23.602085+00:00 bl-cassoop-p11 kernel: [74735.626632]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:49:23.602087+00:00 bl-cassoop-p11 kernel: [74735.626636]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:23.602088+00:00 bl-cassoop-p11 kernel: [74735.626640]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:49:23.602090+00:00 bl-cassoop-p11 kernel: [74735.626643]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:23.602091+00:00 bl-cassoop-p11 kernel: [74735.626647]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:49:23.602093+00:00 bl-cassoop-p11 kernel: [74735.626650]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:49:23.602095+00:00 bl-cassoop-p11 kernel: [74735.626653]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:49:23.602097+00:00 bl-cassoop-p11 kernel: [74735.626656]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:49:23.602099+00:00 bl-cassoop-p11 kernel: [74735.626659]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:49:23.602115+00:00 bl-cassoop-p11 kernel: [74735.626662]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:49:23.602117+00:00 bl-cassoop-p11 kernel: [74735.626666]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:23.602119+00:00 bl-cassoop-p11 kernel: [74735.626670]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:49:23.602120+00:00 bl-cassoop-p11 kernel: [74735.626673]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:23.602122+00:00 bl-cassoop-p11 kernel: [74735.626676]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:49:23.602124+00:00 bl-cassoop-p11 kernel: [74735.626679]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:49:23.602125+00:00 bl-cassoop-p11 kernel: [74735.626682]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:49:23.602127+00:00 bl-cassoop-p11 kernel: [74735.626685]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:49:23.602128+00:00 bl-cassoop-p11 kernel: [74735.626689]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:23.602130+00:00 bl-cassoop-p11 kernel: [74735.626692]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:23.602131+00:00 bl-cassoop-p11 kernel: [74735.626695]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:23.602133+00:00 bl-cassoop-p11 kernel: [74735.626699]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:49:23.602135+00:00 bl-cassoop-p11 kernel: [74735.626702]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:23.602136+00:00 bl-cassoop-p11 kernel: [74735.626705]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:23.602138+00:00 bl-cassoop-p11 kernel: [74735.626708]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:23.602139+00:00 bl-cassoop-p11 kernel: [74735.626712]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:23.602142+00:00 bl-cassoop-p11 kernel: [74735.626713] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:23.625793+00:00 bl-cassoop-p11 kernel: [74735.646546] BUG: soft lockup - CPU#16 stuck for 22s! [kworker/u97:9:42017]
2014-01-18T13:49:23.625800+00:00 bl-cassoop-p11 kernel: [74735.650376] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.625802+00:00 bl-cassoop-p11 kernel: [74735.650400] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.625804+00:00 bl-cassoop-p11 kernel: [74735.650402] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.625805+00:00 bl-cassoop-p11 kernel: [74735.650405] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:49:23.625807+00:00 bl-cassoop-p11 kernel: [74735.650408] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:49:23.625809+00:00 bl-cassoop-p11 kernel: [74735.650410] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:23.625810+00:00 bl-cassoop-p11 kernel: [74735.650413] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:49:23.625812+00:00 bl-cassoop-p11 kernel: [74735.650415] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa98720
2014-01-18T13:49:23.625813+00:00 bl-cassoop-p11 kernel: [74735.650416] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.625815+00:00 bl-cassoop-p11 kernel: [74735.650418] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:49:23.625816+00:00 bl-cassoop-p11 kernel: [74735.650420] R10: 0000000000000004 R11: 0000000000000210 R12: ffffffff81254cb7
2014-01-18T13:49:23.625817+00:00 bl-cassoop-p11 kernel: [74735.650421] R13: ffff880075d0b5c8 R14: 0000000000000292 R15: ffff880075d0b5d8
2014-01-18T13:49:23.625819+00:00 bl-cassoop-p11 kernel: [74735.650423] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.625820+00:00 bl-cassoop-p11 kernel: [74735.650425] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.625822+00:00 bl-cassoop-p11 kernel: [74735.650427] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:23.625823+00:00 bl-cassoop-p11 kernel: [74735.650428] Stack:
2014-01-18T13:49:23.625824+00:00 bl-cassoop-p11 kernel: [74735.650429]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:49:23.625826+00:00 bl-cassoop-p11 kernel: [74735.650435]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:49:23.625859+00:00 bl-cassoop-p11 kernel: [74735.650439]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:49:23.625861+00:00 bl-cassoop-p11 kernel: [74735.650444] Call Trace:
2014-01-18T13:49:23.625863+00:00 bl-cassoop-p11 kernel: [74735.650448]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.625864+00:00 bl-cassoop-p11 kernel: [74735.650452]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.625878+00:00 bl-cassoop-p11 kernel: [74735.650455]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:23.625879+00:00 bl-cassoop-p11 kernel: [74735.650458]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:49:23.625881+00:00 bl-cassoop-p11 kernel: [74735.650461]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:49:23.625883+00:00 bl-cassoop-p11 kernel: [74735.650465]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:49:23.625884+00:00 bl-cassoop-p11 kernel: [74735.650468]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:49:23.625886+00:00 bl-cassoop-p11 kernel: [74735.650471]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:49:23.625887+00:00 bl-cassoop-p11 kernel: [74735.650474]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:49:23.625889+00:00 bl-cassoop-p11 kernel: [74735.650478]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:49:23.625891+00:00 bl-cassoop-p11 kernel: [74735.650482]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:49:23.625892+00:00 bl-cassoop-p11 kernel: [74735.650485]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:49:23.625894+00:00 bl-cassoop-p11 kernel: [74735.650489]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:49:23.625896+00:00 bl-cassoop-p11 kernel: [74735.650493]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:49:23.625898+00:00 bl-cassoop-p11 kernel: [74735.650496]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:49:23.625899+00:00 bl-cassoop-p11 kernel: [74735.650500]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:49:23.625901+00:00 bl-cassoop-p11 kernel: [74735.650503]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:49:23.625902+00:00 bl-cassoop-p11 kernel: [74735.650506]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:49:23.625904+00:00 bl-cassoop-p11 kernel: [74735.650510]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:49:23.625906+00:00 bl-cassoop-p11 kernel: [74735.650513]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:49:23.625907+00:00 bl-cassoop-p11 kernel: [74735.650516]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:49:23.625909+00:00 bl-cassoop-p11 kernel: [74735.650519]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:49:23.625910+00:00 bl-cassoop-p11 kernel: [74735.650523]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:49:23.625912+00:00 bl-cassoop-p11 kernel: [74735.650527]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:49:23.625914+00:00 bl-cassoop-p11 kernel: [74735.650530]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:49:23.625915+00:00 bl-cassoop-p11 kernel: [74735.650533]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:49:23.625917+00:00 bl-cassoop-p11 kernel: [74735.650536]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:23.625918+00:00 bl-cassoop-p11 kernel: [74735.650540]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.625920+00:00 bl-cassoop-p11 kernel: [74735.650545]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:23.625921+00:00 bl-cassoop-p11 kernel: [74735.650548]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.625927+00:00 bl-cassoop-p11 kernel: [74735.650549] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:23.637595+00:00 bl-cassoop-p11 kernel: [74735.658549] BUG: soft lockup - CPU#17 stuck for 23s! [java:42521]
2014-01-18T13:49:23.637602+00:00 bl-cassoop-p11 kernel: [74735.662220] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.637604+00:00 bl-cassoop-p11 kernel: [74735.662242] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.637606+00:00 bl-cassoop-p11 kernel: [74735.662244] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.637607+00:00 bl-cassoop-p11 kernel: [74735.662245] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:49:23.637609+00:00 bl-cassoop-p11 kernel: [74735.662247] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:23.637611+00:00 bl-cassoop-p11 kernel: [74735.662251] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:49:23.637612+00:00 bl-cassoop-p11 kernel: [74735.662253] RAX: 0000000000000006 RBX: ffffffff81d03da0 RCX: ffff88181fa78748
2014-01-18T13:49:23.637614+00:00 bl-cassoop-p11 kernel: [74735.662255] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.637615+00:00 bl-cassoop-p11 kernel: [74735.662256] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:49:23.637617+00:00 bl-cassoop-p11 kernel: [74735.662258] R10: 0000000000000004 R11: 0000000000017960 R12: ffff880cff722548
2014-01-18T13:49:23.637618+00:00 bl-cassoop-p11 kernel: [74735.662259] R13: 0000000000000000 R14: 0000000000000292 R15: ffff882feccbf938
2014-01-18T13:49:23.637619+00:00 bl-cassoop-p11 kernel: [74735.662261] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.637621+00:00 bl-cassoop-p11 kernel: [74735.662263] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.637622+00:00 bl-cassoop-p11 kernel: [74735.662265] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:23.637623+00:00 bl-cassoop-p11 kernel: [74735.662266] Stack:
2014-01-18T13:49:23.637625+00:00 bl-cassoop-p11 kernel: [74735.662267]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:49:23.637626+00:00 bl-cassoop-p11 kernel: [74735.662273]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:49:23.637627+00:00 bl-cassoop-p11 kernel: [74735.662278]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:49:23.637653+00:00 bl-cassoop-p11 kernel: [74735.662282] Call Trace:
2014-01-18T13:49:23.637656+00:00 bl-cassoop-p11 kernel: [74735.662287]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.637657+00:00 bl-cassoop-p11 kernel: [74735.662290]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.637659+00:00 bl-cassoop-p11 kernel: [74735.662294]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:23.637672+00:00 bl-cassoop-p11 kernel: [74735.662297]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:23.637674+00:00 bl-cassoop-p11 kernel: [74735.662301]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:23.637675+00:00 bl-cassoop-p11 kernel: [74735.662304]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:23.637677+00:00 bl-cassoop-p11 kernel: [74735.662309]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:23.637678+00:00 bl-cassoop-p11 kernel: [74735.662312]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:23.637680+00:00 bl-cassoop-p11 kernel: [74735.662315]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:23.637682+00:00 bl-cassoop-p11 kernel: [74735.662318]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:23.637683+00:00 bl-cassoop-p11 kernel: [74735.662322]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:23.637685+00:00 bl-cassoop-p11 kernel: [74735.662325]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:23.637686+00:00 bl-cassoop-p11 kernel: [74735.662328]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:23.637688+00:00 bl-cassoop-p11 kernel: [74735.662332]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:23.637690+00:00 bl-cassoop-p11 kernel: [74735.662335]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:23.637691+00:00 bl-cassoop-p11 kernel: [74735.662339]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:23.637693+00:00 bl-cassoop-p11 kernel: [74735.662344]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:49:23.637694+00:00 bl-cassoop-p11 kernel: [74735.662347]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:49:23.637696+00:00 bl-cassoop-p11 kernel: [74735.662352]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:49:23.637733+00:00 bl-cassoop-p11 kernel: [74735.662355]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:49:23.637736+00:00 bl-cassoop-p11 kernel: [74735.662358]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:49:23.637738+00:00 bl-cassoop-p11 kernel: [74735.662361]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:23.637739+00:00 bl-cassoop-p11 kernel: [74735.662365]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:23.637742+00:00 bl-cassoop-p11 kernel: [74735.662366] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:23.649480+00:00 bl-cassoop-p11 kernel: [74735.670553] BUG: soft lockup - CPU#18 stuck for 23s! [kswapd0:283]
2014-01-18T13:49:23.649488+00:00 bl-cassoop-p11 kernel: [74735.674078] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.649490+00:00 bl-cassoop-p11 kernel: [74735.674100] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.649491+00:00 bl-cassoop-p11 kernel: [74735.674102] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.649493+00:00 bl-cassoop-p11 kernel: [74735.674104] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:49:23.649495+00:00 bl-cassoop-p11 kernel: [74735.674105] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:23.649496+00:00 bl-cassoop-p11 kernel: [74735.674109] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:49:23.649497+00:00 bl-cassoop-p11 kernel: [74735.674111] RAX: 0000000000000282 RBX: 0000000000000000 RCX: 0000000000000001
2014-01-18T13:49:23.649499+00:00 bl-cassoop-p11 kernel: [74735.674113] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:49:23.649500+00:00 bl-cassoop-p11 kernel: [74735.674114] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:49:23.649502+00:00 bl-cassoop-p11 kernel: [74735.674116] R10: 0000000000000001 R11: 0000000000000400 R12: ffffffff810a2fcc
2014-01-18T13:49:23.649509+00:00 bl-cassoop-p11 kernel: [74735.674117] R13: ffff882ff8361698 R14: ffff8817f2191200 R15: ffff882ff8361648
2014-01-18T13:49:23.649510+00:00 bl-cassoop-p11 kernel: [74735.674120] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.649512+00:00 bl-cassoop-p11 kernel: [74735.674121] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.649514+00:00 bl-cassoop-p11 kernel: [74735.674123] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:23.649515+00:00 bl-cassoop-p11 kernel: [74735.674124] Stack:
2014-01-18T13:49:23.649517+00:00 bl-cassoop-p11 kernel: [74735.674126]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:49:23.649518+00:00 bl-cassoop-p11 kernel: [74735.674131]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:49:23.649520+00:00 bl-cassoop-p11 kernel: [74735.674135]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:49:23.649521+00:00 bl-cassoop-p11 kernel: [74735.674140] Call Trace:
2014-01-18T13:49:23.649522+00:00 bl-cassoop-p11 kernel: [74735.674144]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.649524+00:00 bl-cassoop-p11 kernel: [74735.674147]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:23.649525+00:00 bl-cassoop-p11 kernel: [74735.674151]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.649538+00:00 bl-cassoop-p11 kernel: [74735.674154]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.649540+00:00 bl-cassoop-p11 kernel: [74735.674157]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:23.649541+00:00 bl-cassoop-p11 kernel: [74735.674161]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.649543+00:00 bl-cassoop-p11 kernel: [74735.674164]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.649545+00:00 bl-cassoop-p11 kernel: [74735.674168]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:23.649546+00:00 bl-cassoop-p11 kernel: [74735.674172]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:23.649548+00:00 bl-cassoop-p11 kernel: [74735.674175]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:23.649550+00:00 bl-cassoop-p11 kernel: [74735.674178]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:49:23.649551+00:00 bl-cassoop-p11 kernel: [74735.674182]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:23.649553+00:00 bl-cassoop-p11 kernel: [74735.674185]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:23.649555+00:00 bl-cassoop-p11 kernel: [74735.674189]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:23.649557+00:00 bl-cassoop-p11 kernel: [74735.674192]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:23.649558+00:00 bl-cassoop-p11 kernel: [74735.674195]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:49:23.649560+00:00 bl-cassoop-p11 kernel: [74735.674199]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:23.649561+00:00 bl-cassoop-p11 kernel: [74735.674203]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:23.649563+00:00 bl-cassoop-p11 kernel: [74735.674206]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:23.649565+00:00 bl-cassoop-p11 kernel: [74735.674209]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:23.649566+00:00 bl-cassoop-p11 kernel: [74735.674213]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:23.649568+00:00 bl-cassoop-p11 kernel: [74735.674216]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:23.649570+00:00 bl-cassoop-p11 kernel: [74735.674220]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:23.649571+00:00 bl-cassoop-p11 kernel: [74735.674223]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:23.649573+00:00 bl-cassoop-p11 kernel: [74735.674226]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:23.649574+00:00 bl-cassoop-p11 kernel: [74735.674230]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:23.649576+00:00 bl-cassoop-p11 kernel: [74735.674233]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:23.649577+00:00 bl-cassoop-p11 kernel: [74735.674237]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.649579+00:00 bl-cassoop-p11 kernel: [74735.674241]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:23.649581+00:00 bl-cassoop-p11 kernel: [74735.674244]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:23.649585+00:00 bl-cassoop-p11 kernel: [74735.674245] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:23.673219+00:00 bl-cassoop-p11 kernel: [74735.694564] BUG: soft lockup - CPU#20 stuck for 23s! [jsvc:42019]
2014-01-18T13:49:23.673227+00:00 bl-cassoop-p11 kernel: [74735.697938] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.673229+00:00 bl-cassoop-p11 kernel: [74735.697966] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.673231+00:00 bl-cassoop-p11 kernel: [74735.697968] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.673232+00:00 bl-cassoop-p11 kernel: [74735.697969] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:49:23.673234+00:00 bl-cassoop-p11 kernel: [74735.697970] RIP: 0010:[<ffffffff810cb9a4>]  [<ffffffff810cb9a4>] smp_call_function_many+0x274/0x2d0
2014-01-18T13:49:23.673235+00:00 bl-cassoop-p11 kernel: [74735.697974] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:49:23.673237+00:00 bl-cassoop-p11 kernel: [74735.697975] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa387c0
2014-01-18T13:49:23.673238+00:00 bl-cassoop-p11 kernel: [74735.697976] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:23.673240+00:00 bl-cassoop-p11 kernel: [74735.697977] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:49:23.673241+00:00 bl-cassoop-p11 kernel: [74735.697978] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:49:23.673243+00:00 bl-cassoop-p11 kernel: [74735.697979] R13: ffff8801111b3c48 R14: 0000000000000292 R15: ffff8801111b3c78
2014-01-18T13:49:23.673244+00:00 bl-cassoop-p11 kernel: [74735.697980] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.673245+00:00 bl-cassoop-p11 kernel: [74735.697982] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.673247+00:00 bl-cassoop-p11 kernel: [74735.697983] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:23.673248+00:00 bl-cassoop-p11 kernel: [74735.697984] Stack:
2014-01-18T13:49:23.673249+00:00 bl-cassoop-p11 kernel: [74735.697985]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:49:23.673251+00:00 bl-cassoop-p11 kernel: [74735.697988]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:49:23.673295+00:00 bl-cassoop-p11 kernel: [74735.697991]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:49:23.673297+00:00 bl-cassoop-p11 kernel: [74735.697994] Call Trace:
2014-01-18T13:49:23.673299+00:00 bl-cassoop-p11 kernel: [74735.697997]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.673301+00:00 bl-cassoop-p11 kernel: [74735.698000]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:23.673302+00:00 bl-cassoop-p11 kernel: [74735.698003]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:23.673315+00:00 bl-cassoop-p11 kernel: [74735.698006]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:23.673317+00:00 bl-cassoop-p11 kernel: [74735.698008]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:23.673318+00:00 bl-cassoop-p11 kernel: [74735.698010]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:23.673320+00:00 bl-cassoop-p11 kernel: [74735.698013]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:23.673321+00:00 bl-cassoop-p11 kernel: [74735.698015]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:23.673323+00:00 bl-cassoop-p11 kernel: [74735.698018]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:23.673325+00:00 bl-cassoop-p11 kernel: [74735.698019] Code: 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 74 c4 <eb> f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff 48 83
2014-01-18T13:49:23.757147+00:00 bl-cassoop-p11 kernel: [74735.778593] BUG: soft lockup - CPU#27 stuck for 23s! [java:42434]
2014-01-18T13:49:23.757155+00:00 bl-cassoop-p11 kernel: [74735.781799] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:23.757158+00:00 bl-cassoop-p11 kernel: [74735.781817] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:23.757159+00:00 bl-cassoop-p11 kernel: [74735.781818] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:23.757161+00:00 bl-cassoop-p11 kernel: [74735.781819] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:49:23.757163+00:00 bl-cassoop-p11 kernel: [74735.781820] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:23.757164+00:00 bl-cassoop-p11 kernel: [74735.781824] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:49:23.757166+00:00 bl-cassoop-p11 kernel: [74735.781825] RAX: 0000000000000030 RBX: ffffffff810a3789 RCX: 0000000000000010
2014-01-18T13:49:23.757167+00:00 bl-cassoop-p11 kernel: [74735.781826] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:49:23.757169+00:00 bl-cassoop-p11 kernel: [74735.781827] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:49:23.757170+00:00 bl-cassoop-p11 kernel: [74735.781828] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:23.757172+00:00 bl-cassoop-p11 kernel: [74735.781829] R13: 000000000000000e R14: ffffffff810a108e R15: ffff881887d733b8
2014-01-18T13:49:23.757173+00:00 bl-cassoop-p11 kernel: [74735.781831] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:23.757174+00:00 bl-cassoop-p11 kernel: [74735.781832] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:23.757176+00:00 bl-cassoop-p11 kernel: [74735.781833] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:23.757177+00:00 bl-cassoop-p11 kernel: [74735.781834] Stack:
2014-01-18T13:49:23.757178+00:00 bl-cassoop-p11 kernel: [74735.781835]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:49:23.757181+00:00 bl-cassoop-p11 kernel: [74735.781838]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:49:23.757183+00:00 bl-cassoop-p11 kernel: [74735.781841]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:49:23.757184+00:00 bl-cassoop-p11 kernel: [74735.781844] Call Trace:
2014-01-18T13:49:23.757185+00:00 bl-cassoop-p11 kernel: [74735.781848]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.757187+00:00 bl-cassoop-p11 kernel: [74735.781850]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:23.757188+00:00 bl-cassoop-p11 kernel: [74735.781852]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.757190+00:00 bl-cassoop-p11 kernel: [74735.781854]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:23.757191+00:00 bl-cassoop-p11 kernel: [74735.781856]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:23.757193+00:00 bl-cassoop-p11 kernel: [74735.781859]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:23.757195+00:00 bl-cassoop-p11 kernel: [74735.781862]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:23.757208+00:00 bl-cassoop-p11 kernel: [74735.781871]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:23.757210+00:00 bl-cassoop-p11 kernel: [74735.781874]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:23.757212+00:00 bl-cassoop-p11 kernel: [74735.781878]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:49:23.757213+00:00 bl-cassoop-p11 kernel: [74735.781880]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:23.757215+00:00 bl-cassoop-p11 kernel: [74735.781883]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:23.757216+00:00 bl-cassoop-p11 kernel: [74735.781885]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:23.757218+00:00 bl-cassoop-p11 kernel: [74735.781888]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:23.757220+00:00 bl-cassoop-p11 kernel: [74735.781890]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:23.757221+00:00 bl-cassoop-p11 kernel: [74735.781893]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:23.757223+00:00 bl-cassoop-p11 kernel: [74735.781896]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:23.757224+00:00 bl-cassoop-p11 kernel: [74735.781898]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:23.757226+00:00 bl-cassoop-p11 kernel: [74735.781902]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:23.757227+00:00 bl-cassoop-p11 kernel: [74735.781905]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:23.757229+00:00 bl-cassoop-p11 kernel: [74735.781907]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:23.757231+00:00 bl-cassoop-p11 kernel: [74735.781910]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:23.757233+00:00 bl-cassoop-p11 kernel: [74735.781912]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:49:23.757234+00:00 bl-cassoop-p11 kernel: [74735.781916]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:23.757236+00:00 bl-cassoop-p11 kernel: [74735.781919]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:23.757237+00:00 bl-cassoop-p11 kernel: [74735.781922]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:23.757239+00:00 bl-cassoop-p11 kernel: [74735.781924]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:23.757241+00:00 bl-cassoop-p11 kernel: [74735.781927]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:23.757243+00:00 bl-cassoop-p11 kernel: [74735.781930]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:23.757244+00:00 bl-cassoop-p11 kernel: [74735.781932]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:23.757246+00:00 bl-cassoop-p11 kernel: [74735.781934]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:23.757247+00:00 bl-cassoop-p11 kernel: [74735.781937]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:23.757258+00:00 bl-cassoop-p11 kernel: [74735.781939]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:23.757260+00:00 bl-cassoop-p11 kernel: [74735.781941]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:23.757261+00:00 bl-cassoop-p11 kernel: [74735.781945]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:23.757263+00:00 bl-cassoop-p11 kernel: [74735.781949]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:23.757265+00:00 bl-cassoop-p11 kernel: [74735.781952]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:23.757266+00:00 bl-cassoop-p11 kernel: [74735.781954]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:23.757268+00:00 bl-cassoop-p11 kernel: [74735.781957]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:23.757269+00:00 bl-cassoop-p11 kernel: [74735.781959]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:23.757271+00:00 bl-cassoop-p11 kernel: [74735.781961]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:23.757273+00:00 bl-cassoop-p11 kernel: [74735.781962] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:27.863574+00:00 bl-cassoop-p11 kernel: [74739.876089] INFO: rcu_sched self-detected stall on CPU { 10}  (t=15001 jiffies g=3080496 c=3080495 q=0)
2014-01-18T13:49:27.863591+00:00 bl-cassoop-p11 kernel: [74739.880093] INFO: rcu_sched detected stalls on CPUs/tasks: { 10} (detected by 27, t=15002 jiffies, g=3080496, c=3080495, q=0)
2014-01-18T13:49:27.863593+00:00 bl-cassoop-p11 kernel: [74739.880094] sending NMI to all CPUs:
2014-01-18T13:49:27.863594+00:00 bl-cassoop-p11 kernel: [74739.880097] NMI backtrace for cpu 0
2014-01-18T13:49:27.863597+00:00 bl-cassoop-p11 kernel: [74739.880100] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863598+00:00 bl-cassoop-p11 kernel: [74739.880101] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863600+00:00 bl-cassoop-p11 kernel: [74739.880102] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:49:27.863603+00:00 bl-cassoop-p11 kernel: [74739.880105] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.863604+00:00 bl-cassoop-p11 kernel: [74739.880106] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:49:27.863606+00:00 bl-cassoop-p11 kernel: [74739.880107] RAX: 0000000000000008 RBX: ffff88181fa15088 RCX: ffff88181fa97868
2014-01-18T13:49:27.863608+00:00 bl-cassoop-p11 kernel: [74739.880108] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.863609+00:00 bl-cassoop-p11 kernel: [74739.880109] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:49:27.863611+00:00 bl-cassoop-p11 kernel: [74739.880110] R10: 0000000000000004 R11: 0000000000000206 R12: ffff88181fa15080
2014-01-18T13:49:27.863613+00:00 bl-cassoop-p11 kernel: [74739.880111] R13: ffffffff8105ae90 R14: ffff880045ccfd50 R15: 0000000000015040
2014-01-18T13:49:27.863614+00:00 bl-cassoop-p11 kernel: [74739.880112] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.863616+00:00 bl-cassoop-p11 kernel: [74739.880113] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.863617+00:00 bl-cassoop-p11 kernel: [74739.880114] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:49:27.863618+00:00 bl-cassoop-p11 kernel: [74739.880114] Stack:
2014-01-18T13:49:27.863620+00:00 bl-cassoop-p11 kernel: [74739.880120]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:49:27.863622+00:00 bl-cassoop-p11 kernel: [74739.880123]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:49:27.863623+00:00 bl-cassoop-p11 kernel: [74739.880126]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:49:27.863624+00:00 bl-cassoop-p11 kernel: [74739.880127] Call Trace:
2014-01-18T13:49:27.863626+00:00 bl-cassoop-p11 kernel: [74739.880130]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.863628+00:00 bl-cassoop-p11 kernel: [74739.880133]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.863629+00:00 bl-cassoop-p11 kernel: [74739.880136]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:27.863631+00:00 bl-cassoop-p11 kernel: [74739.880139]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:27.863633+00:00 bl-cassoop-p11 kernel: [74739.880140]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:27.863635+00:00 bl-cassoop-p11 kernel: [74739.880143]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:27.863636+00:00 bl-cassoop-p11 kernel: [74739.880146]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:27.863638+00:00 bl-cassoop-p11 kernel: [74739.880148]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:27.863639+00:00 bl-cassoop-p11 kernel: [74739.880151]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.863642+00:00 bl-cassoop-p11 kernel: [74739.880170] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.863644+00:00 bl-cassoop-p11 kernel: [74739.880197] NMI backtrace for cpu 1
2014-01-18T13:49:27.863646+00:00 bl-cassoop-p11 kernel: [74739.880199] CPU: 1 PID: 5830 Comm: collectd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863648+00:00 bl-cassoop-p11 kernel: [74739.880200] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863649+00:00 bl-cassoop-p11 kernel: [74739.880201] task: ffff882ff0912ee0 ti: ffff882ffa6f6000 task.ti: ffff882ffa6f6000
2014-01-18T13:49:27.863651+00:00 bl-cassoop-p11 kernel: [74739.880204] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:27.863653+00:00 bl-cassoop-p11 kernel: [74739.880205] RSP: 0018:ffff882ffa6f7d08  EFLAGS: 00000202
2014-01-18T13:49:27.863660+00:00 bl-cassoop-p11 kernel: [74739.880206] RAX: 0000000000000286 RBX: ffff882ffa6f7d70 RCX: 0000000000000001
2014-01-18T13:49:27.863662+00:00 bl-cassoop-p11 kernel: [74739.880207] RDX: ffff88181fa38680 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:27.863664+00:00 bl-cassoop-p11 kernel: [74739.880208] RBP: ffff882ffa6f7d48 R08: 0000000000000000 R09: 000000000000730e
2014-01-18T13:49:27.863665+00:00 bl-cassoop-p11 kernel: [74739.880209] R10: 0000000000003708 R11: 0000000000000293 R12: 0000000000000002
2014-01-18T13:49:27.863667+00:00 bl-cassoop-p11 kernel: [74739.880209] R13: ffff88181fa35040 R14: ffff88181fa35050 R15: 0000000000000001
2014-01-18T13:49:27.863668+00:00 bl-cassoop-p11 kernel: [74739.880211] FS:  00007f2a8a89f700(0000) GS:ffff88301f200000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.863670+00:00 bl-cassoop-p11 kernel: [74739.880212] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.863671+00:00 bl-cassoop-p11 kernel: [74739.880212] CR2: 00007f2a92182000 CR3: 0000002ff0b0e000 CR4: 00000000000407e0
2014-01-18T13:49:27.863672+00:00 bl-cassoop-p11 kernel: [74739.880213] Stack:
2014-01-18T13:49:27.863674+00:00 bl-cassoop-p11 kernel: [74739.880218]  ffff882ff7817000 ffff88181fa386a8 00000007f2a9217e 0000000000000002
2014-01-18T13:49:27.863675+00:00 bl-cassoop-p11 kernel: [74739.880222]  ffffffff8139a810 0000000000000001 ffffffff81d03da0 00007f2a92182000
2014-01-18T13:49:27.863677+00:00 bl-cassoop-p11 kernel: [74739.880225]  ffff882ffa6f7db8 ffffffff810cb5a5 ffff882ff0912ee0 0000000000000001
2014-01-18T13:49:27.863678+00:00 bl-cassoop-p11 kernel: [74739.880225] Call Trace:
2014-01-18T13:49:27.863737+00:00 bl-cassoop-p11 kernel: [74739.880231]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:27.863739+00:00 bl-cassoop-p11 kernel: [74739.880233]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.863741+00:00 bl-cassoop-p11 kernel: [74739.880236]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:27.863743+00:00 bl-cassoop-p11 kernel: [74739.880239]  [<ffffffff8139ab4e>] rdmsr_on_cpu+0x4e/0x70
2014-01-18T13:49:27.863745+00:00 bl-cassoop-p11 kernel: [74739.880244]  [<ffffffffa001b62b>] show_temp+0xab/0xf0 [coretemp]
2014-01-18T13:49:27.863746+00:00 bl-cassoop-p11 kernel: [74739.880249]  [<ffffffff81490f20>] dev_attr_show+0x20/0x60
2014-01-18T13:49:27.863748+00:00 bl-cassoop-p11 kernel: [74739.880252]  [<ffffffff8114d37e>] ? __get_free_pages+0xe/0x40
2014-01-18T13:49:27.863750+00:00 bl-cassoop-p11 kernel: [74739.880258]  [<ffffffff81227e86>] fill_read_buffer.isra.9+0x66/0xf0
2014-01-18T13:49:27.863751+00:00 bl-cassoop-p11 kernel: [74739.880260]  [<ffffffff81227fc0>] sysfs_read_file+0xb0/0xf0
2014-01-18T13:49:27.863753+00:00 bl-cassoop-p11 kernel: [74739.880263]  [<ffffffff811b42e4>] vfs_read+0xb4/0x180
2014-01-18T13:49:27.863755+00:00 bl-cassoop-p11 kernel: [74739.880266]  [<ffffffff811b4562>] SyS_read+0x52/0xa0
2014-01-18T13:49:27.863756+00:00 bl-cassoop-p11 kernel: [74739.880269]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.863759+00:00 bl-cassoop-p11 kernel: [74739.880287] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:27.863761+00:00 bl-cassoop-p11 kernel: [74739.880296] NMI backtrace for cpu 2
2014-01-18T13:49:27.863762+00:00 bl-cassoop-p11 kernel: [74739.880298] CPU: 2 PID: 42018 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863764+00:00 bl-cassoop-p11 kernel: [74739.880299] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863766+00:00 bl-cassoop-p11 kernel: [74739.880300] task: ffff881881ad4650 ti: ffff88010dc50000 task.ti: ffff88010dc50000
2014-01-18T13:49:27.863768+00:00 bl-cassoop-p11 kernel: [74739.880303] RIP: 0010:[<ffffffff813817cd>]  [<ffffffff813817cd>] __write_lock_failed+0xd/0x20
2014-01-18T13:49:27.863769+00:00 bl-cassoop-p11 kernel: [74739.880304] RSP: 0018:ffff88010dc51e68  EFLAGS: 00000087
2014-01-18T13:49:27.863771+00:00 bl-cassoop-p11 kernel: [74739.880305] RAX: ffff88010dc51ea0 RBX: ffff881881ad4650 RCX: 0000000000000001
2014-01-18T13:49:27.863773+00:00 bl-cassoop-p11 kernel: [74739.880306] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.863774+00:00 bl-cassoop-p11 kernel: [74739.880307] RBP: ffff88010dc51e68 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.863776+00:00 bl-cassoop-p11 kernel: [74739.880307] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:27.863777+00:00 bl-cassoop-p11 kernel: [74739.880308] R13: ffff881881ad4650 R14: 0000000000000000 R15: 0000000000000003
2014-01-18T13:49:27.863779+00:00 bl-cassoop-p11 kernel: [74739.880310] FS:  0000000000000000(0000) GS:ffff88181fa20000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.863780+00:00 bl-cassoop-p11 kernel: [74739.880311] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.863782+00:00 bl-cassoop-p11 kernel: [74739.880311] CR2: 00007eff09a77530 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.863783+00:00 bl-cassoop-p11 kernel: [74739.880312] Stack:
2014-01-18T13:49:27.863785+00:00 bl-cassoop-p11 kernel: [74739.880317]  ffff88010dc51e78 ffffffff8174787e ffff88010dc51ee8 ffffffff810676e5
2014-01-18T13:49:27.863786+00:00 bl-cassoop-p11 kernel: [74739.880320]  ffff88010dc51f08 ffff88010dc51ea0 ffff881881ad4650 ffff88010dc51ea0
2014-01-18T13:49:27.863788+00:00 bl-cassoop-p11 kernel: [74739.880323]  ffff88010dc51ea0 0000000100000000 0001c78d00000001 ffff881881ad4650
2014-01-18T13:49:27.863789+00:00 bl-cassoop-p11 kernel: [74739.880324] Call Trace:
2014-01-18T13:49:27.863790+00:00 bl-cassoop-p11 kernel: [74739.880327]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.863792+00:00 bl-cassoop-p11 kernel: [74739.880329]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.863794+00:00 bl-cassoop-p11 kernel: [74739.880332]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.863796+00:00 bl-cassoop-p11 kernel: [74739.880334]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.863797+00:00 bl-cassoop-p11 kernel: [74739.880337]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:27.863799+00:00 bl-cassoop-p11 kernel: [74739.880339]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.863801+00:00 bl-cassoop-p11 kernel: [74739.880358] Code: 01 31 c0 66 66 90 c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 <81> 3f 00 00 10 00 75 f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48
2014-01-18T13:49:27.863802+00:00 bl-cassoop-p11 kernel: [74739.880395] NMI backtrace for cpu 3
2014-01-18T13:49:27.863804+00:00 bl-cassoop-p11 kernel: [74739.880398] CPU: 3 PID: 5299 Comm: nrpe Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863807+00:00 bl-cassoop-p11 kernel: [74739.880398] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863808+00:00 bl-cassoop-p11 kernel: [74739.880400] task: ffff882ff17d8000 ti: ffff882ff0376000 task.ti: ffff882ff0376000
2014-01-18T13:49:27.863811+00:00 bl-cassoop-p11 kernel: [74739.880403] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.863812+00:00 bl-cassoop-p11 kernel: [74739.880403] RSP: 0018:ffff882ff0377e10  EFLAGS: 00000087
2014-01-18T13:49:27.863813+00:00 bl-cassoop-p11 kernel: [74739.880404] RAX: 000000000000a638 RBX: ffff881880f29770 RCX: 0000000000000000
2014-01-18T13:49:27.863815+00:00 bl-cassoop-p11 kernel: [74739.880405] RDX: 0000000000000011 RSI: 0000000000000006 RDI: ffffffff81c06040
2014-01-18T13:49:27.863816+00:00 bl-cassoop-p11 kernel: [74739.880405] RBP: ffff882ff0377e10 R08: ffff8823fe8717b0 R09: 0000000000000040
2014-01-18T13:49:27.863818+00:00 bl-cassoop-p11 kernel: [74739.880406] R10: ffff8823fe8717c0 R11: 0000000000000000 R12: 0000000001200011
2014-01-18T13:49:27.863820+00:00 bl-cassoop-p11 kernel: [74739.880406] R13: ffff881880f29b00 R14: 0000000000000000 R15: 00007f451b60b9d0
2014-01-18T13:49:27.863885+00:00 bl-cassoop-p11 kernel: [74739.880407] FS:  00007f451b60b700(0000) GS:ffff88301f220000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.863886+00:00 bl-cassoop-p11 kernel: [74739.880408] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.863888+00:00 bl-cassoop-p11 kernel: [74739.880408] CR2: 000000079d884bfc CR3: 0000002ff7997000 CR4: 00000000000407e0
2014-01-18T13:49:27.863889+00:00 bl-cassoop-p11 kernel: [74739.880409] Stack:
2014-01-18T13:49:27.863890+00:00 bl-cassoop-p11 kernel: [74739.880411]  ffff882ff0377e20 ffffffff8174787e ffff882ff0377ea0 ffffffff81063fda
2014-01-18T13:49:27.863892+00:00 bl-cassoop-p11 kernel: [74739.880413]  fffffffffffffff5 0000000000000000 ffff881880f29a80 0000000000000005
2014-01-18T13:49:27.863894+00:00 bl-cassoop-p11 kernel: [74739.880415]  0000000000000000 ffff8823fe871780 ffffffff811d1165 00000000f0377f58
2014-01-18T13:49:27.863895+00:00 bl-cassoop-p11 kernel: [74739.880416] Call Trace:
2014-01-18T13:49:27.863896+00:00 bl-cassoop-p11 kernel: [74739.880418]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.863898+00:00 bl-cassoop-p11 kernel: [74739.880420]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:27.863899+00:00 bl-cassoop-p11 kernel: [74739.880423]  [<ffffffff811d1165>] ? fd_install+0x25/0x30
2014-01-18T13:49:27.863901+00:00 bl-cassoop-p11 kernel: [74739.880426]  [<ffffffff8162162a>] ? SYSC_accept4+0x17a/0x240
2014-01-18T13:49:27.863903+00:00 bl-cassoop-p11 kernel: [74739.880428]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:27.863904+00:00 bl-cassoop-p11 kernel: [74739.880429]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:27.863906+00:00 bl-cassoop-p11 kernel: [74739.880431]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:27.863908+00:00 bl-cassoop-p11 kernel: [74739.880433]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:27.863909+00:00 bl-cassoop-p11 kernel: [74739.880435]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.863912+00:00 bl-cassoop-p11 kernel: [74739.880447] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.863913+00:00 bl-cassoop-p11 kernel: [74739.880499] NMI backtrace for cpu 4
2014-01-18T13:49:27.863915+00:00 bl-cassoop-p11 kernel: [74739.880501] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863917+00:00 bl-cassoop-p11 kernel: [74739.880502] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863918+00:00 bl-cassoop-p11 kernel: [74739.880503] task: ffff8817fad80000 ti: ffff8817fad88000 task.ti: ffff8817fad88000
2014-01-18T13:49:27.863920+00:00 bl-cassoop-p11 kernel: [74739.880508] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.863922+00:00 bl-cassoop-p11 kernel: [74739.880509] RSP: 0018:ffff8817fad89da8  EFLAGS: 00000046
2014-01-18T13:49:27.863923+00:00 bl-cassoop-p11 kernel: [74739.880510] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.863925+00:00 bl-cassoop-p11 kernel: [74739.880511] RDX: 0000000000000000 RSI: ffff8817fad89fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.863926+00:00 bl-cassoop-p11 kernel: [74739.880512] RBP: ffff8817fad89dd8 R08: 000000000000d3a9 R09: 000000002bdd6040
2014-01-18T13:49:27.863928+00:00 bl-cassoop-p11 kernel: [74739.880512] R10: 0000000000001d93 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.863929+00:00 bl-cassoop-p11 kernel: [74739.880513] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.863931+00:00 bl-cassoop-p11 kernel: [74739.880515] FS:  0000000000000000(0000) GS:ffff88181fa40000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.863932+00:00 bl-cassoop-p11 kernel: [74739.880516] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.863934+00:00 bl-cassoop-p11 kernel: [74739.880517] CR2: 000000008b06c0d0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.863935+00:00 bl-cassoop-p11 kernel: [74739.880517] Stack:
2014-01-18T13:49:27.863936+00:00 bl-cassoop-p11 kernel: [74739.880521]  ffff8817fad89dd8 00000004810bc824 ffffe8e800240320 000043f36452da01
2014-01-18T13:49:27.863938+00:00 bl-cassoop-p11 kernel: [74739.880524]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad89e38 ffffffff815e5a9f
2014-01-18T13:49:27.863939+00:00 bl-cassoop-p11 kernel: [74739.880528]  0000000000000000 0000000006f5438d 0000000000000000 0000000006f5438d
2014-01-18T13:49:27.863940+00:00 bl-cassoop-p11 kernel: [74739.880528] Call Trace:
2014-01-18T13:49:27.863942+00:00 bl-cassoop-p11 kernel: [74739.880532]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.863944+00:00 bl-cassoop-p11 kernel: [74739.880535]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.863945+00:00 bl-cassoop-p11 kernel: [74739.880538]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.863947+00:00 bl-cassoop-p11 kernel: [74739.880541]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.863948+00:00 bl-cassoop-p11 kernel: [74739.880544]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.863950+00:00 bl-cassoop-p11 kernel: [74739.880548]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.863953+00:00 bl-cassoop-p11 kernel: [74739.880567] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.863954+00:00 bl-cassoop-p11 kernel: [74739.880616] NMI backtrace for cpu 5
2014-01-18T13:49:27.863956+00:00 bl-cassoop-p11 kernel: [74739.880618] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.863957+00:00 bl-cassoop-p11 kernel: [74739.880619] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.863959+00:00 bl-cassoop-p11 kernel: [74739.880620] task: ffff8817fad81770 ti: ffff8817fad8a000 task.ti: ffff8817fad8a000
2014-01-18T13:49:27.863961+00:00 bl-cassoop-p11 kernel: [74739.880624] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.863962+00:00 bl-cassoop-p11 kernel: [74739.880624] RSP: 0018:ffff8817fad8bda8  EFLAGS: 00000046
2014-01-18T13:49:27.863964+00:00 bl-cassoop-p11 kernel: [74739.880625] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.864061+00:00 bl-cassoop-p11 kernel: [74739.880625] RDX: 0000000000000000 RSI: ffff8817fad8bfd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.864063+00:00 bl-cassoop-p11 kernel: [74739.880626] RBP: ffff8817fad8bdd8 R08: 0000000000005409 R09: 000000000109b729
2014-01-18T13:49:27.864065+00:00 bl-cassoop-p11 kernel: [74739.880627] R10: 0000000000001a25 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.864066+00:00 bl-cassoop-p11 kernel: [74739.880627] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.864068+00:00 bl-cassoop-p11 kernel: [74739.880628] FS:  0000000000000000(0000) GS:ffff88301f240000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864069+00:00 bl-cassoop-p11 kernel: [74739.880629] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864071+00:00 bl-cassoop-p11 kernel: [74739.880630] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.864072+00:00 bl-cassoop-p11 kernel: [74739.880630] Stack:
2014-01-18T13:49:27.864074+00:00 bl-cassoop-p11 kernel: [74739.880633]  ffff8817fad8bdd8 00000005810bc824 ffffe8ffffa40320 000043f361adda53
2014-01-18T13:49:27.864075+00:00 bl-cassoop-p11 kernel: [74739.880635]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad8be38 ffffffff815e5a9f
2014-01-18T13:49:27.864077+00:00 bl-cassoop-p11 kernel: [74739.880637]  0000000000000000 000000000cb6bd45 0000000000000000 000000000cb6bd45
2014-01-18T13:49:27.864078+00:00 bl-cassoop-p11 kernel: [74739.880637] Call Trace:
2014-01-18T13:49:27.864079+00:00 bl-cassoop-p11 kernel: [74739.880641]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.864081+00:00 bl-cassoop-p11 kernel: [74739.880642]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.864083+00:00 bl-cassoop-p11 kernel: [74739.880645]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.864085+00:00 bl-cassoop-p11 kernel: [74739.880647]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.864086+00:00 bl-cassoop-p11 kernel: [74739.880649]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.864088+00:00 bl-cassoop-p11 kernel: [74739.880652]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.864090+00:00 bl-cassoop-p11 kernel: [74739.880663] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.864092+00:00 bl-cassoop-p11 kernel: [74739.880693] NMI backtrace for cpu 6
2014-01-18T13:49:27.864094+00:00 bl-cassoop-p11 kernel: [74739.880696] CPU: 6 PID: 5135 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864096+00:00 bl-cassoop-p11 kernel: [74739.880697] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864097+00:00 bl-cassoop-p11 kernel: [74739.880698] task: ffff8817f9b4ddc0 ti: ffff8817f927c000 task.ti: ffff8817f927c000
2014-01-18T13:49:27.864099+00:00 bl-cassoop-p11 kernel: [74739.880701] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.864101+00:00 bl-cassoop-p11 kernel: [74739.880702] RSP: 0018:ffff8817f927de10  EFLAGS: 00000087
2014-01-18T13:49:27.864102+00:00 bl-cassoop-p11 kernel: [74739.880703] RAX: 0000000000001382 RBX: ffff88171d6bddc0 RCX: 0000000000000000
2014-01-18T13:49:27.864104+00:00 bl-cassoop-p11 kernel: [74739.880703] RDX: ffff8817f3792ee0 RSI: 0000000000000006 RDI: ffffffff81c06040
2014-01-18T13:49:27.864105+00:00 bl-cassoop-p11 kernel: [74739.880704] RBP: ffff8817f927de10 R08: ffff8817f7bd2430 R09: 0000000000000040
2014-01-18T13:49:27.864107+00:00 bl-cassoop-p11 kernel: [74739.880704] R10: ffff8817f7bd2440 R11: e000000000000000 R12: 00000000003d0f00
2014-01-18T13:49:27.864108+00:00 bl-cassoop-p11 kernel: [74739.880705] R13: ffff88171d6be150 R14: 0000000000000000 R15: 00007f58eeeee9d0
2014-01-18T13:49:27.864110+00:00 bl-cassoop-p11 kernel: [74739.880706] FS:  00007f58fc73e700(0000) GS:ffff88181fa60000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864111+00:00 bl-cassoop-p11 kernel: [74739.880707] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864113+00:00 bl-cassoop-p11 kernel: [74739.880707] CR2: 00007fd1e9a6b218 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.864114+00:00 bl-cassoop-p11 kernel: [74739.880708] Stack:
2014-01-18T13:49:27.864124+00:00 bl-cassoop-p11 kernel: [74739.880710]  ffff8817f927de20 ffffffff8174787e ffff8817f927dea0 ffffffff81063fda
2014-01-18T13:49:27.864126+00:00 bl-cassoop-p11 kernel: [74739.880712]  fffffffffffffff5 0000000000010000 ffff88171d6be0d0 00000000a35ce200
2014-01-18T13:49:27.864127+00:00 bl-cassoop-p11 kernel: [74739.880714]  00007f58eeeedff0 ffff8817f7bd2400 ffffffff8167e2e0 00000000f927df58
2014-01-18T13:49:27.864128+00:00 bl-cassoop-p11 kernel: [74739.880715] Call Trace:
2014-01-18T13:49:27.864130+00:00 bl-cassoop-p11 kernel: [74739.880717]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.864132+00:00 bl-cassoop-p11 kernel: [74739.880719]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:27.864133+00:00 bl-cassoop-p11 kernel: [74739.880721]  [<ffffffff8167e2e0>] ? do_tcp_setsockopt.isra.29.part.30+0x60/0x720
2014-01-18T13:49:27.864135+00:00 bl-cassoop-p11 kernel: [74739.880723]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:27.864137+00:00 bl-cassoop-p11 kernel: [74739.880724]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:27.864138+00:00 bl-cassoop-p11 kernel: [74739.880726]  [<ffffffff811c4ae8>] ? setfl+0x118/0x170
2014-01-18T13:49:27.864140+00:00 bl-cassoop-p11 kernel: [74739.880727]  [<ffffffff811c5038>] ? do_fcntl+0x258/0x340
2014-01-18T13:49:27.864141+00:00 bl-cassoop-p11 kernel: [74739.880729]  [<ffffffff8167eab3>] ? tcp_setsockopt+0x33/0x40
2014-01-18T13:49:27.864143+00:00 bl-cassoop-p11 kernel: [74739.880731]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:27.864144+00:00 bl-cassoop-p11 kernel: [74739.880733]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:27.864146+00:00 bl-cassoop-p11 kernel: [74739.880735]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.864149+00:00 bl-cassoop-p11 kernel: [74739.880747] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.864150+00:00 bl-cassoop-p11 kernel: [74739.880815] NMI backtrace for cpu 7
2014-01-18T13:49:27.864211+00:00 bl-cassoop-p11 kernel: [74739.880817] CPU: 7 PID: 0 Comm: swapper/7 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864213+00:00 bl-cassoop-p11 kernel: [74739.880818] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864215+00:00 bl-cassoop-p11 kernel: [74739.880819] task: ffff8817fad84650 ti: ffff8817fad8e000 task.ti: ffff8817fad8e000
2014-01-18T13:49:27.864217+00:00 bl-cassoop-p11 kernel: [74739.880822] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.864218+00:00 bl-cassoop-p11 kernel: [74739.880822] RSP: 0000:ffff8817fad8fda8  EFLAGS: 00000046
2014-01-18T13:49:27.864219+00:00 bl-cassoop-p11 kernel: [74739.880823] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.864221+00:00 bl-cassoop-p11 kernel: [74739.880824] RDX: 0000000000000000 RSI: ffff8817fad8ffd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.864223+00:00 bl-cassoop-p11 kernel: [74739.880825] RBP: ffff8817fad8fdd8 R08: 0000000000006315 R09: 0000000005dcf504
2014-01-18T13:49:27.864224+00:00 bl-cassoop-p11 kernel: [74739.880825] R10: 0000000000001f2e R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.864226+00:00 bl-cassoop-p11 kernel: [74739.880826] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.864227+00:00 bl-cassoop-p11 kernel: [74739.880827] FS:  0000000000000000(0000) GS:ffff88301f260000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864229+00:00 bl-cassoop-p11 kernel: [74739.880828] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864230+00:00 bl-cassoop-p11 kernel: [74739.880828] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.864231+00:00 bl-cassoop-p11 kernel: [74739.880829] Stack:
2014-01-18T13:49:27.864233+00:00 bl-cassoop-p11 kernel: [74739.880832]  ffff8817fad8fdd8 00000007810bc824 ffffe8ffffa60320 000043f361adf848
2014-01-18T13:49:27.864235+00:00 bl-cassoop-p11 kernel: [74739.880834]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad8fe38 ffffffff815e5a9f
2014-01-18T13:49:27.864236+00:00 bl-cassoop-p11 kernel: [74739.880836]  0000000000000000 0000000003b5f7f3 0000000000000000 0000000003b5f7f3
2014-01-18T13:49:27.864237+00:00 bl-cassoop-p11 kernel: [74739.880836] Call Trace:
2014-01-18T13:49:27.864239+00:00 bl-cassoop-p11 kernel: [74739.880839]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.864241+00:00 bl-cassoop-p11 kernel: [74739.880841]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.864242+00:00 bl-cassoop-p11 kernel: [74739.880843]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.864244+00:00 bl-cassoop-p11 kernel: [74739.880845]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.864246+00:00 bl-cassoop-p11 kernel: [74739.880847]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.864247+00:00 bl-cassoop-p11 kernel: [74739.880849]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.864250+00:00 bl-cassoop-p11 kernel: [74739.880861] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.864251+00:00 bl-cassoop-p11 kernel: [74739.880892] NMI backtrace for cpu 8
2014-01-18T13:49:27.864253+00:00 bl-cassoop-p11 kernel: [74739.880893] CPU: 8 PID: 42507 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864255+00:00 bl-cassoop-p11 kernel: [74739.880894] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864256+00:00 bl-cassoop-p11 kernel: [74739.880895] task: ffff881176ee4650 ti: ffff880016e20000 task.ti: ffff880016e20000
2014-01-18T13:49:27.864258+00:00 bl-cassoop-p11 kernel: [74739.880898] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.864260+00:00 bl-cassoop-p11 kernel: [74739.880898] RSP: 0018:ffff880016e21e68  EFLAGS: 00000087
2014-01-18T13:49:27.864261+00:00 bl-cassoop-p11 kernel: [74739.880899] RAX: ffff880016e21ea0 RBX: ffff881176ee4650 RCX: 0000000000000001
2014-01-18T13:49:27.864263+00:00 bl-cassoop-p11 kernel: [74739.880900] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.864264+00:00 bl-cassoop-p11 kernel: [74739.880900] RBP: ffff880016e21e68 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.864265+00:00 bl-cassoop-p11 kernel: [74739.880901] R10: 0000000000000001 R11: 0000000000000400 R12: 0000000000000000
2014-01-18T13:49:27.864273+00:00 bl-cassoop-p11 kernel: [74739.880901] R13: ffff881176ee4650 R14: 0000000000000000 R15: 0000000000000003
2014-01-18T13:49:27.864275+00:00 bl-cassoop-p11 kernel: [74739.880902] FS:  00007f58eeeee700(0000) GS:ffff88181fa80000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864276+00:00 bl-cassoop-p11 kernel: [74739.880903] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864278+00:00 bl-cassoop-p11 kernel: [74739.880904] CR2: 000000078be993a8 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.864279+00:00 bl-cassoop-p11 kernel: [74739.880904] Stack:
2014-01-18T13:49:27.864280+00:00 bl-cassoop-p11 kernel: [74739.880907]  ffff880016e21e78 ffffffff8174787e ffff880016e21ee8 ffffffff810676e5
2014-01-18T13:49:27.864282+00:00 bl-cassoop-p11 kernel: [74739.880909]  ffff880016e21f08 ffff880016e21ea0 ffff881176ee4650 ffff880016e21ea0
2014-01-18T13:49:27.864285+00:00 bl-cassoop-p11 kernel: [74739.880911]  ffff880016e21ea0 0000000100000000 0001e00200000001 ffff881176ee4650
2014-01-18T13:49:27.864286+00:00 bl-cassoop-p11 kernel: [74739.880911] Call Trace:
2014-01-18T13:49:27.864288+00:00 bl-cassoop-p11 kernel: [74739.880914]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.864290+00:00 bl-cassoop-p11 kernel: [74739.880916]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.864291+00:00 bl-cassoop-p11 kernel: [74739.880917]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.864293+00:00 bl-cassoop-p11 kernel: [74739.880919]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.864294+00:00 bl-cassoop-p11 kernel: [74739.880921]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:27.864296+00:00 bl-cassoop-p11 kernel: [74739.880923]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.864416+00:00 bl-cassoop-p11 kernel: [74739.880934] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.864418+00:00 bl-cassoop-p11 kernel: [74739.880995] NMI backtrace for cpu 10
2014-01-18T13:49:27.864420+00:00 bl-cassoop-p11 kernel: [74739.880997] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864421+00:00 bl-cassoop-p11 kernel: [74739.880997] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864423+00:00 bl-cassoop-p11 kernel: [74739.880998] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:49:27.864425+00:00 bl-cassoop-p11 kernel: [74739.881003] RIP: 0010:[<ffffffff813ebdf8>]  [<ffffffff813ebdf8>] cfb_imageblit+0x2c8/0x4d0
2014-01-18T13:49:27.864426+00:00 bl-cassoop-p11 kernel: [74739.881003] RSP: 0000:ffff88181faa37e8  EFLAGS: 00000046
2014-01-18T13:49:27.864430+00:00 bl-cassoop-p11 kernel: [74739.881004] RAX: ffffc900292fbb50 RBX: ffffc900292fbb54 RCX: 0000000000000003
2014-01-18T13:49:27.864431+00:00 bl-cassoop-p11 kernel: [74739.881004] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
2014-01-18T13:49:27.864433+00:00 bl-cassoop-p11 kernel: [74739.881005] RBP: ffff88181faa3868 R08: 0000000000000000 R09: ffff882ff730b5a5
2014-01-18T13:49:27.864434+00:00 bl-cassoop-p11 kernel: [74739.881006] R10: 0000000000000001 R11: ffffffff8185c888 R12: ffff882ff730b594
2014-01-18T13:49:27.864436+00:00 bl-cassoop-p11 kernel: [74739.881006] R13: ffffc900292fbba0 R14: 000000000000000b R15: ffffc900292fb920
2014-01-18T13:49:27.864437+00:00 bl-cassoop-p11 kernel: [74739.881007] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864438+00:00 bl-cassoop-p11 kernel: [74739.881008] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864446+00:00 bl-cassoop-p11 kernel: [74739.881009] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:27.864447+00:00 bl-cassoop-p11 kernel: [74739.881009] Stack:
2014-01-18T13:49:27.864449+00:00 bl-cassoop-p11 kernel: [74739.881011]  000000000000000a ffffffff81ef8fa0 ffff88181faa3858 ffffffff8137f340
2014-01-18T13:49:27.864450+00:00 bl-cassoop-p11 kernel: [74739.881013]  000000a01faa3818 ffffffff00000020 0000000000000a00 0000000000000280
2014-01-18T13:49:27.864452+00:00 bl-cassoop-p11 kernel: [74739.881015]  0000000000000014 ffffc900292f8720 0000000000000028 ffff88181f418000
2014-01-18T13:49:27.864453+00:00 bl-cassoop-p11 kernel: [74739.881016] Call Trace:
2014-01-18T13:49:27.864454+00:00 bl-cassoop-p11 kernel: [74739.881017]  <IRQ>
2014-01-18T13:49:27.864455+00:00 bl-cassoop-p11 kernel: [74739.881019]  [<ffffffff8137f340>] ? sprintf+0x40/0x50
2014-01-18T13:49:27.864459+00:00 bl-cassoop-p11 kernel: [74739.881021]  [<ffffffff813e9235>] bit_putcs+0x2e5/0x570
2014-01-18T13:49:27.864460+00:00 bl-cassoop-p11 kernel: [74739.881023]  [<ffffffff8137d7a5>] ? number.isra.1+0x355/0x390
2014-01-18T13:49:27.864462+00:00 bl-cassoop-p11 kernel: [74739.881025]  [<ffffffff813e8f03>] ? bit_cursor+0x5f3/0x640
2014-01-18T13:49:27.864464+00:00 bl-cassoop-p11 kernel: [74739.881027]  [<ffffffff813e3260>] ? get_color.isra.16+0x40/0x160
2014-01-18T13:49:27.864465+00:00 bl-cassoop-p11 kernel: [74739.881029]  [<ffffffff813e4f97>] fbcon_putcs+0x137/0x170
2014-01-18T13:49:27.864467+00:00 bl-cassoop-p11 kernel: [74739.881031]  [<ffffffff813e8f50>] ? bit_cursor+0x640/0x640
2014-01-18T13:49:27.864468+00:00 bl-cassoop-p11 kernel: [74739.881033]  [<ffffffff813e55bc>] fbcon_redraw.isra.24+0x16c/0x1e0
2014-01-18T13:49:27.864472+00:00 bl-cassoop-p11 kernel: [74739.881035]  [<ffffffff813e661d>] fbcon_scroll+0x21d/0xda0
2014-01-18T13:49:27.864473+00:00 bl-cassoop-p11 kernel: [74739.881038]  [<ffffffff81462b63>] scrup+0x113/0x120
2014-01-18T13:49:27.864475+00:00 bl-cassoop-p11 kernel: [74739.881040]  [<ffffffff81462cd0>] lf+0x70/0x80
2014-01-18T13:49:27.864476+00:00 bl-cassoop-p11 kernel: [74739.881042]  [<ffffffff81463fb8>] vt_console_print+0x218/0x3b0
2014-01-18T13:49:27.864478+00:00 bl-cassoop-p11 kernel: [74739.881044]  [<ffffffff810a4253>] ? update_group_power+0xf3/0x100
2014-01-18T13:49:27.864480+00:00 bl-cassoop-p11 kernel: [74739.881046]  [<ffffffff810b7d23>] call_console_drivers.constprop.17+0xa3/0x110
2014-01-18T13:49:27.864481+00:00 bl-cassoop-p11 kernel: [74739.881048]  [<ffffffff810b7f1d>] console_cont_flush.constprop.16+0x8d/0xb0
2014-01-18T13:49:27.864483+00:00 bl-cassoop-p11 kernel: [74739.881050]  [<ffffffff810b8eb8>] console_unlock.part.12+0x28/0x270
2014-01-18T13:49:27.864486+00:00 bl-cassoop-p11 kernel: [74739.881052]  [<ffffffff810b911a>] console_unlock+0x1a/0x30
2014-01-18T13:49:27.864488+00:00 bl-cassoop-p11 kernel: [74739.881054]  [<ffffffff810b9605>] vprintk_emit+0x255/0x490
2014-01-18T13:49:27.864489+00:00 bl-cassoop-p11 kernel: [74739.881056]  [<ffffffff8172f673>] printk+0x61/0x63
2014-01-18T13:49:27.864491+00:00 bl-cassoop-p11 kernel: [74739.881059]  [<ffffffff81105aff>] print_cpu_stall+0xcf/0x140
2014-01-18T13:49:27.864493+00:00 bl-cassoop-p11 kernel: [74739.881061]  [<ffffffff81106507>] __rcu_pending+0x77/0x1d0
2014-01-18T13:49:27.864494+00:00 bl-cassoop-p11 kernel: [74739.881063]  [<ffffffff81107e4d>] rcu_check_callbacks+0xed/0x1b0
2014-01-18T13:49:27.864496+00:00 bl-cassoop-p11 kernel: [74739.881065]  [<ffffffff810741b8>] update_process_times+0x48/0x80
2014-01-18T13:49:27.864499+00:00 bl-cassoop-p11 kernel: [74739.881068]  [<ffffffff810c59e3>] tick_sched_handle.isra.10+0x33/0x70
2014-01-18T13:49:27.864501+00:00 bl-cassoop-p11 kernel: [74739.881070]  [<ffffffff810c5b0c>] tick_sched_timer+0x4c/0x80
2014-01-18T13:49:27.864502+00:00 bl-cassoop-p11 kernel: [74739.881071]  [<ffffffff8108c8b8>] __run_hrtimer+0x78/0x230
2014-01-18T13:49:27.864504+00:00 bl-cassoop-p11 kernel: [74739.881073]  [<ffffffff810c5ac0>] ? tick_nohz_handler+0xa0/0xa0
2014-01-18T13:49:27.864506+00:00 bl-cassoop-p11 kernel: [74739.881075]  [<ffffffff8108d0bf>] hrtimer_interrupt+0xff/0x240
2014-01-18T13:49:27.864507+00:00 bl-cassoop-p11 kernel: [74739.881077]  [<ffffffff810437bb>] local_apic_timer_interrupt+0x3b/0x60
2014-01-18T13:49:27.864509+00:00 bl-cassoop-p11 kernel: [74739.881079]  [<ffffffff81752d25>] smp_apic_timer_interrupt+0x45/0x60
2014-01-18T13:49:27.864511+00:00 bl-cassoop-p11 kernel: [74739.881081]  [<ffffffff8175169d>] apic_timer_interrupt+0x6d/0x80
2014-01-18T13:49:27.864513+00:00 bl-cassoop-p11 kernel: [74739.881083]  <EOI>
2014-01-18T13:49:27.864515+00:00 bl-cassoop-p11 kernel: [74739.881084]  [<ffffffff81731068>] ? oom_kill_process.part.6+0xae/0x2cf
2014-01-18T13:49:27.864517+00:00 bl-cassoop-p11 kernel: [74739.881086]  [<ffffffff8173105d>] ? oom_kill_process.part.6+0xa3/0x2cf
2014-01-18T13:49:27.864518+00:00 bl-cassoop-p11 kernel: [74739.881088]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:49:27.864619+00:00 bl-cassoop-p11 kernel: [74739.881090]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:49:27.864620+00:00 bl-cassoop-p11 kernel: [74739.881091]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:49:27.864622+00:00 bl-cassoop-p11 kernel: [74739.881093]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:49:27.864626+00:00 bl-cassoop-p11 kernel: [74739.881095]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:49:27.864627+00:00 bl-cassoop-p11 kernel: [74739.881097]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:49:27.864629+00:00 bl-cassoop-p11 kernel: [74739.881099]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.864630+00:00 bl-cassoop-p11 kernel: [74739.881101]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.864632+00:00 bl-cassoop-p11 kernel: [74739.881103]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:27.864634+00:00 bl-cassoop-p11 kernel: [74739.881105]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:27.864636+00:00 bl-cassoop-p11 kernel: [74739.881107]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:27.864637+00:00 bl-cassoop-p11 kernel: [74739.881109]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.864642+00:00 bl-cassoop-p11 kernel: [74739.881111]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:27.864644+00:00 bl-cassoop-p11 kernel: [74739.881112]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.864645+00:00 bl-cassoop-p11 kernel: [74739.881114]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:49:27.864647+00:00 bl-cassoop-p11 kernel: [74739.881115]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.864648+00:00 bl-cassoop-p11 kernel: [74739.881118]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.864650+00:00 bl-cassoop-p11 kernel: [74739.881119]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:27.864651+00:00 bl-cassoop-p11 kernel: [74739.881121]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:27.864653+00:00 bl-cassoop-p11 kernel: [74739.881122]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.864656+00:00 bl-cassoop-p11 kernel: [74739.881124]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:27.864658+00:00 bl-cassoop-p11 kernel: [74739.881126]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.864660+00:00 bl-cassoop-p11 kernel: [74739.881128]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.864662+00:00 bl-cassoop-p11 kernel: [74739.881139] Code: 6d b8 4c 89 f8 4d 89 e1 b9 08 00 00 00 4d 01 fd eb 28 90 41 0f be 11 44 29 d1 48 8d 58 04 d3 fa 21 fa 41 8b 14 93 44 21 c2 31 f2 <89> 10 85 c9 75 06 49 83 c1 01 b1 08 48 89 d8 4c 39 e8 75 d4 4c
2014-01-18T13:49:27.864664+00:00 bl-cassoop-p11 kernel: [74739.881141] NMI backtrace for cpu 9
2014-01-18T13:49:27.864665+00:00 bl-cassoop-p11 kernel: [74739.881144] CPU: 9 PID: 0 Comm: swapper/9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864669+00:00 bl-cassoop-p11 kernel: [74739.881144] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864670+00:00 bl-cassoop-p11 kernel: [74739.881145] task: ffff8817fad98000 ti: ffff8817fad92000 task.ti: ffff8817fad92000
2014-01-18T13:49:27.864672+00:00 bl-cassoop-p11 kernel: [74739.881150] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.864674+00:00 bl-cassoop-p11 kernel: [74739.881151] RSP: 0018:ffff8817fad93da8  EFLAGS: 00000046
2014-01-18T13:49:27.864675+00:00 bl-cassoop-p11 kernel: [74739.881152] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.864677+00:00 bl-cassoop-p11 kernel: [74739.881152] RDX: 0000000000000000 RSI: ffff8817fad93fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.864678+00:00 bl-cassoop-p11 kernel: [74739.881153] RBP: ffff8817fad93dd8 R08: 00000000000062ad R09: 000000000342e4d9
2014-01-18T13:49:27.864680+00:00 bl-cassoop-p11 kernel: [74739.881153] R10: 0000000000001b57 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.864683+00:00 bl-cassoop-p11 kernel: [74739.881154] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.864684+00:00 bl-cassoop-p11 kernel: [74739.881155] FS:  0000000000000000(0000) GS:ffff88301f280000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864686+00:00 bl-cassoop-p11 kernel: [74739.881156] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864687+00:00 bl-cassoop-p11 kernel: [74739.881156] CR2: 00007f54cc04ff90 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.864688+00:00 bl-cassoop-p11 kernel: [74739.881157] Stack:
2014-01-18T13:49:27.864690+00:00 bl-cassoop-p11 kernel: [74739.881159]  ffff8817fad93dd8 00000009810bc824 ffffe8ffffa80320 000043f35b8418e6
2014-01-18T13:49:27.864691+00:00 bl-cassoop-p11 kernel: [74739.881161]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad93e38 ffffffff815e5a9f
2014-01-18T13:49:27.864695+00:00 bl-cassoop-p11 kernel: [74739.881163]  0000000000000000 000000001c1d6b6e 0000000000000000 000000001c1d6b6e
2014-01-18T13:49:27.864696+00:00 bl-cassoop-p11 kernel: [74739.881164] Call Trace:
2014-01-18T13:49:27.864697+00:00 bl-cassoop-p11 kernel: [74739.881166]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.864699+00:00 bl-cassoop-p11 kernel: [74739.881168]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.864701+00:00 bl-cassoop-p11 kernel: [74739.881170]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.864702+00:00 bl-cassoop-p11 kernel: [74739.881172]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.864704+00:00 bl-cassoop-p11 kernel: [74739.881174]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.864705+00:00 bl-cassoop-p11 kernel: [74739.881175]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.864710+00:00 bl-cassoop-p11 kernel: [74739.881187] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.864711+00:00 bl-cassoop-p11 kernel: [74739.881189] NMI backtrace for cpu 11
2014-01-18T13:49:27.864713+00:00 bl-cassoop-p11 kernel: [74739.881191] CPU: 11 PID: 0 Comm: swapper/11 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864714+00:00 bl-cassoop-p11 kernel: [74739.881192] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864779+00:00 bl-cassoop-p11 kernel: [74739.881194] task: ffff8817fad9aee0 ti: ffff8817fad96000 task.ti: ffff8817fad96000
2014-01-18T13:49:27.864781+00:00 bl-cassoop-p11 kernel: [74739.881197] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.864782+00:00 bl-cassoop-p11 kernel: [74739.881198] RSP: 0018:ffff8817fad97da8  EFLAGS: 00000046
2014-01-18T13:49:27.864786+00:00 bl-cassoop-p11 kernel: [74739.881199] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.864787+00:00 bl-cassoop-p11 kernel: [74739.881200] RDX: 0000000000000000 RSI: ffff8817fad97fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.864789+00:00 bl-cassoop-p11 kernel: [74739.881200] RBP: ffff8817fad97dd8 R08: 0000000000022091 R09: 0000000002cf5444
2014-01-18T13:49:27.864790+00:00 bl-cassoop-p11 kernel: [74739.881201] R10: 000000000000169c R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.864791+00:00 bl-cassoop-p11 kernel: [74739.881202] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.864793+00:00 bl-cassoop-p11 kernel: [74739.881203] FS:  0000000000000000(0000) GS:ffff88301f2a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864794+00:00 bl-cassoop-p11 kernel: [74739.881204] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864798+00:00 bl-cassoop-p11 kernel: [74739.881205] CR2: 0000000001bc3000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.864799+00:00 bl-cassoop-p11 kernel: [74739.881206] Stack:
2014-01-18T13:49:27.864800+00:00 bl-cassoop-p11 kernel: [74739.881209]  ffff8817fad97dd8 0000000b810bc824 ffffe8ffffaa0320 000043f35b04a261
2014-01-18T13:49:27.864802+00:00 bl-cassoop-p11 kernel: [74739.881213]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad97e38 ffffffff815e5a9f
2014-01-18T13:49:27.864803+00:00 bl-cassoop-p11 kernel: [74739.881216]  0000000000000000 000000001dd4d15f 0000000000000000 000000001dd4d15f
2014-01-18T13:49:27.864804+00:00 bl-cassoop-p11 kernel: [74739.881216] Call Trace:
2014-01-18T13:49:27.864806+00:00 bl-cassoop-p11 kernel: [74739.881221]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.864807+00:00 bl-cassoop-p11 kernel: [74739.881223]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.864811+00:00 bl-cassoop-p11 kernel: [74739.881225]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.864813+00:00 bl-cassoop-p11 kernel: [74739.881228]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.864814+00:00 bl-cassoop-p11 kernel: [74739.881230]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.864816+00:00 bl-cassoop-p11 kernel: [74739.881233]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.864818+00:00 bl-cassoop-p11 kernel: [74739.881251] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.864820+00:00 bl-cassoop-p11 kernel: [74739.881253] NMI backtrace for cpu 12
2014-01-18T13:49:27.864821+00:00 bl-cassoop-p11 kernel: [74739.881255] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864825+00:00 bl-cassoop-p11 kernel: [74739.881255] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864827+00:00 bl-cassoop-p11 kernel: [74739.881256] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:49:27.864829+00:00 bl-cassoop-p11 kernel: [74739.881259] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.864830+00:00 bl-cassoop-p11 kernel: [74739.881260] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:49:27.864832+00:00 bl-cassoop-p11 kernel: [74739.881260] RAX: 0000000000000002 RBX: ffff88181fad5088 RCX: ffff88181fa38680
2014-01-18T13:49:27.864833+00:00 bl-cassoop-p11 kernel: [74739.881261] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.864835+00:00 bl-cassoop-p11 kernel: [74739.881261] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:49:27.864838+00:00 bl-cassoop-p11 kernel: [74739.881262] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff88181fad5080
2014-01-18T13:49:27.864839+00:00 bl-cassoop-p11 kernel: [74739.881263] R13: ffffffff8105ae90 R14: ffff8817f8e45a10 R15: 0000000000015040
2014-01-18T13:49:27.864841+00:00 bl-cassoop-p11 kernel: [74739.881264] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864842+00:00 bl-cassoop-p11 kernel: [74739.881265] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864844+00:00 bl-cassoop-p11 kernel: [74739.881265] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.864845+00:00 bl-cassoop-p11 kernel: [74739.881266] Stack:
2014-01-18T13:49:27.864846+00:00 bl-cassoop-p11 kernel: [74739.881268]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:49:27.864848+00:00 bl-cassoop-p11 kernel: [74739.881270]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:49:27.864851+00:00 bl-cassoop-p11 kernel: [74739.881272]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:49:27.864852+00:00 bl-cassoop-p11 kernel: [74739.881273] Call Trace:
2014-01-18T13:49:27.864854+00:00 bl-cassoop-p11 kernel: [74739.881276]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.864856+00:00 bl-cassoop-p11 kernel: [74739.881277]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.864857+00:00 bl-cassoop-p11 kernel: [74739.881279]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.864859+00:00 bl-cassoop-p11 kernel: [74739.881281]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:27.864860+00:00 bl-cassoop-p11 kernel: [74739.881283]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:27.864864+00:00 bl-cassoop-p11 kernel: [74739.881285]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:27.864865+00:00 bl-cassoop-p11 kernel: [74739.881286]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:27.864867+00:00 bl-cassoop-p11 kernel: [74739.881288]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:27.864869+00:00 bl-cassoop-p11 kernel: [74739.881289]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:27.864870+00:00 bl-cassoop-p11 kernel: [74739.881291]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:27.864872+00:00 bl-cassoop-p11 kernel: [74739.881292]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:27.864940+00:00 bl-cassoop-p11 kernel: [74739.881294]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:27.864941+00:00 bl-cassoop-p11 kernel: [74739.881295]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:27.864945+00:00 bl-cassoop-p11 kernel: [74739.881297]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:49:27.864946+00:00 bl-cassoop-p11 kernel: [74739.881299]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.864948+00:00 bl-cassoop-p11 kernel: [74739.881300]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.864949+00:00 bl-cassoop-p11 kernel: [74739.881302]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.864951+00:00 bl-cassoop-p11 kernel: [74739.881304]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.864953+00:00 bl-cassoop-p11 kernel: [74739.881316] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.864955+00:00 bl-cassoop-p11 kernel: [74739.881317] NMI backtrace for cpu 13
2014-01-18T13:49:27.864959+00:00 bl-cassoop-p11 kernel: [74739.881319] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.864961+00:00 bl-cassoop-p11 kernel: [74739.881319] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.864963+00:00 bl-cassoop-p11 kernel: [74739.881320] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:49:27.864965+00:00 bl-cassoop-p11 kernel: [74739.881322] RIP: 0010:[<ffffffff810cb9a4>]  [<ffffffff810cb9a4>] smp_call_function_many+0x274/0x2d0
2014-01-18T13:49:27.864966+00:00 bl-cassoop-p11 kernel: [74739.881323] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:49:27.864968+00:00 bl-cassoop-p11 kernel: [74739.881324] RAX: 0000000000000002 RBX: ffff88301f2d5088 RCX: ffff88181fa386a8
2014-01-18T13:49:27.864969+00:00 bl-cassoop-p11 kernel: [74739.881325] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.864972+00:00 bl-cassoop-p11 kernel: [74739.881325] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:49:27.864974+00:00 bl-cassoop-p11 kernel: [74739.881326] R10: 0000000000000004 R11: 0000000000000000 R12: ffff88301f2d5080
2014-01-18T13:49:27.864975+00:00 bl-cassoop-p11 kernel: [74739.881327] R13: ffffffff8105ae90 R14: ffff882ff8363820 R15: 0000000000015040
2014-01-18T13:49:27.864977+00:00 bl-cassoop-p11 kernel: [74739.881328] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.864978+00:00 bl-cassoop-p11 kernel: [74739.881328] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.864980+00:00 bl-cassoop-p11 kernel: [74739.881329] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.864981+00:00 bl-cassoop-p11 kernel: [74739.881329] Stack:
2014-01-18T13:49:27.864984+00:00 bl-cassoop-p11 kernel: [74739.881331]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:49:27.864986+00:00 bl-cassoop-p11 kernel: [74739.881333]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:49:27.864987+00:00 bl-cassoop-p11 kernel: [74739.881335]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:49:27.864988+00:00 bl-cassoop-p11 kernel: [74739.881336] Call Trace:
2014-01-18T13:49:27.864990+00:00 bl-cassoop-p11 kernel: [74739.881339]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.864991+00:00 bl-cassoop-p11 kernel: [74739.881341]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.864993+00:00 bl-cassoop-p11 kernel: [74739.881343]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.864995+00:00 bl-cassoop-p11 kernel: [74739.881344]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:27.864998+00:00 bl-cassoop-p11 kernel: [74739.881346]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:27.865000+00:00 bl-cassoop-p11 kernel: [74739.881348]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:27.865001+00:00 bl-cassoop-p11 kernel: [74739.881350]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:27.865003+00:00 bl-cassoop-p11 kernel: [74739.881352]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:27.865005+00:00 bl-cassoop-p11 kernel: [74739.881354]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.865006+00:00 bl-cassoop-p11 kernel: [74739.881356]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.865008+00:00 bl-cassoop-p11 kernel: [74739.881358]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.865010+00:00 bl-cassoop-p11 kernel: [74739.881359]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.865013+00:00 bl-cassoop-p11 kernel: [74739.881361]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.865015+00:00 bl-cassoop-p11 kernel: [74739.881363]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:27.865017+00:00 bl-cassoop-p11 kernel: [74739.881365]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:27.865018+00:00 bl-cassoop-p11 kernel: [74739.881366]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:27.865020+00:00 bl-cassoop-p11 kernel: [74739.881368]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:27.865021+00:00 bl-cassoop-p11 kernel: [74739.881370]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:27.865023+00:00 bl-cassoop-p11 kernel: [74739.881372]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.865026+00:00 bl-cassoop-p11 kernel: [74739.881374]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865028+00:00 bl-cassoop-p11 kernel: [74739.881376]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.865029+00:00 bl-cassoop-p11 kernel: [74739.881378]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865032+00:00 bl-cassoop-p11 kernel: [74739.881389] Code: 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 74 c4 <eb> f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff 48 83
2014-01-18T13:49:27.865033+00:00 bl-cassoop-p11 kernel: [74739.881391] NMI backtrace for cpu 14
2014-01-18T13:49:27.865035+00:00 bl-cassoop-p11 kernel: [74739.881393] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865107+00:00 bl-cassoop-p11 kernel: [74739.881394] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865111+00:00 bl-cassoop-p11 kernel: [74739.881395] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:49:27.865113+00:00 bl-cassoop-p11 kernel: [74739.881398] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.865114+00:00 bl-cassoop-p11 kernel: [74739.881399] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:49:27.865116+00:00 bl-cassoop-p11 kernel: [74739.881400] RAX: 0000000000000008 RBX: ffff88181faf5088 RCX: ffff88181fa986d0
2014-01-18T13:49:27.865117+00:00 bl-cassoop-p11 kernel: [74739.881400] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.865118+00:00 bl-cassoop-p11 kernel: [74739.881401] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:49:27.865120+00:00 bl-cassoop-p11 kernel: [74739.881402] R10: 0000000000000004 R11: 0000000000000058 R12: ffff88181faf5080
2014-01-18T13:49:27.865123+00:00 bl-cassoop-p11 kernel: [74739.881403] R13: ffffffff8105ae90 R14: ffff88195e365180 R15: 0000000000015040
2014-01-18T13:49:27.865125+00:00 bl-cassoop-p11 kernel: [74739.881405] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865126+00:00 bl-cassoop-p11 kernel: [74739.881406] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865128+00:00 bl-cassoop-p11 kernel: [74739.881407] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:49:27.865129+00:00 bl-cassoop-p11 kernel: [74739.881407] Stack:
2014-01-18T13:49:27.865130+00:00 bl-cassoop-p11 kernel: [74739.881411]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:49:27.865132+00:00 bl-cassoop-p11 kernel: [74739.881414]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:49:27.865133+00:00 bl-cassoop-p11 kernel: [74739.881417]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:49:27.865136+00:00 bl-cassoop-p11 kernel: [74739.881418] Call Trace:
2014-01-18T13:49:27.865138+00:00 bl-cassoop-p11 kernel: [74739.881421]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.865140+00:00 bl-cassoop-p11 kernel: [74739.881424]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.865141+00:00 bl-cassoop-p11 kernel: [74739.881426]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.865143+00:00 bl-cassoop-p11 kernel: [74739.881429]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:27.865144+00:00 bl-cassoop-p11 kernel: [74739.881431]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:27.865146+00:00 bl-cassoop-p11 kernel: [74739.881434]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:27.865149+00:00 bl-cassoop-p11 kernel: [74739.881436]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:27.865151+00:00 bl-cassoop-p11 kernel: [74739.881439]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:27.865152+00:00 bl-cassoop-p11 kernel: [74739.881442]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.865154+00:00 bl-cassoop-p11 kernel: [74739.881445]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.865155+00:00 bl-cassoop-p11 kernel: [74739.881447]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.865157+00:00 bl-cassoop-p11 kernel: [74739.881450]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:27.865158+00:00 bl-cassoop-p11 kernel: [74739.881452]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:27.865160+00:00 bl-cassoop-p11 kernel: [74739.881455]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:27.865167+00:00 bl-cassoop-p11 kernel: [74739.881457]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:27.865169+00:00 bl-cassoop-p11 kernel: [74739.881460]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:27.865171+00:00 bl-cassoop-p11 kernel: [74739.881463]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:27.865172+00:00 bl-cassoop-p11 kernel: [74739.881466]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:27.865174+00:00 bl-cassoop-p11 kernel: [74739.881469]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.865176+00:00 bl-cassoop-p11 kernel: [74739.881471]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.865177+00:00 bl-cassoop-p11 kernel: [74739.881474]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:49:27.865181+00:00 bl-cassoop-p11 kernel: [74739.881476]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:49:27.865182+00:00 bl-cassoop-p11 kernel: [74739.881479]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:49:27.865184+00:00 bl-cassoop-p11 kernel: [74739.881481]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:49:27.865186+00:00 bl-cassoop-p11 kernel: [74739.881484]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:27.865187+00:00 bl-cassoop-p11 kernel: [74739.881487]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:49:27.865189+00:00 bl-cassoop-p11 kernel: [74739.881489]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:27.865191+00:00 bl-cassoop-p11 kernel: [74739.881492]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:49:27.865192+00:00 bl-cassoop-p11 kernel: [74739.881494]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:49:27.865196+00:00 bl-cassoop-p11 kernel: [74739.881496]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:49:27.865197+00:00 bl-cassoop-p11 kernel: [74739.881498]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:49:27.865199+00:00 bl-cassoop-p11 kernel: [74739.881501]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:49:27.865200+00:00 bl-cassoop-p11 kernel: [74739.881503]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:49:27.865202+00:00 bl-cassoop-p11 kernel: [74739.881506]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.865204+00:00 bl-cassoop-p11 kernel: [74739.881509]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:49:27.865205+00:00 bl-cassoop-p11 kernel: [74739.881511]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.865208+00:00 bl-cassoop-p11 kernel: [74739.881513]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:49:27.865210+00:00 bl-cassoop-p11 kernel: [74739.881516]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:49:27.865212+00:00 bl-cassoop-p11 kernel: [74739.881518]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:49:27.865291+00:00 bl-cassoop-p11 kernel: [74739.881520]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:49:27.865293+00:00 bl-cassoop-p11 kernel: [74739.881522]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.865295+00:00 bl-cassoop-p11 kernel: [74739.881524]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.865296+00:00 bl-cassoop-p11 kernel: [74739.881527]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.865298+00:00 bl-cassoop-p11 kernel: [74739.881529]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:49:27.865302+00:00 bl-cassoop-p11 kernel: [74739.881531]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:27.865304+00:00 bl-cassoop-p11 kernel: [74739.881534]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:27.865305+00:00 bl-cassoop-p11 kernel: [74739.881536]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.865307+00:00 bl-cassoop-p11 kernel: [74739.881539]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.865309+00:00 bl-cassoop-p11 kernel: [74739.881558] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.865311+00:00 bl-cassoop-p11 kernel: [74739.881559] NMI backtrace for cpu 16
2014-01-18T13:49:27.865312+00:00 bl-cassoop-p11 kernel: [74739.881561] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865316+00:00 bl-cassoop-p11 kernel: [74739.881562] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865317+00:00 bl-cassoop-p11 kernel: [74739.881565] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:49:27.865319+00:00 bl-cassoop-p11 kernel: [74739.881566] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:49:27.865321+00:00 bl-cassoop-p11 kernel: [74739.881568] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.865322+00:00 bl-cassoop-p11 kernel: [74739.881569] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:49:27.865324+00:00 bl-cassoop-p11 kernel: [74739.881570] RAX: 0000000000000008 RBX: ffff88181fb15088 RCX: ffff88181fa98720
2014-01-18T13:49:27.865325+00:00 bl-cassoop-p11 kernel: [74739.881571] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.865329+00:00 bl-cassoop-p11 kernel: [74739.881572] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:49:27.865330+00:00 bl-cassoop-p11 kernel: [74739.881573] R10: 0000000000000004 R11: 0000000000000210 R12: ffff88181fb15080
2014-01-18T13:49:27.865332+00:00 bl-cassoop-p11 kernel: [74739.881574] R13: ffffffff8105ae90 R14: ffff880075d0b6b0 R15: 0000000000015040
2014-01-18T13:49:27.865333+00:00 bl-cassoop-p11 kernel: [74739.881575] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865334+00:00 bl-cassoop-p11 kernel: [74739.881576] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865336+00:00 bl-cassoop-p11 kernel: [74739.881577] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.865337+00:00 bl-cassoop-p11 kernel: [74739.881577] Stack:
2014-01-18T13:49:27.865338+00:00 bl-cassoop-p11 kernel: [74739.881581]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:49:27.865342+00:00 bl-cassoop-p11 kernel: [74739.881584]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:49:27.865343+00:00 bl-cassoop-p11 kernel: [74739.881587]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:49:27.865344+00:00 bl-cassoop-p11 kernel: [74739.881588] Call Trace:
2014-01-18T13:49:27.865346+00:00 bl-cassoop-p11 kernel: [74739.881591]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.865347+00:00 bl-cassoop-p11 kernel: [74739.881593]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.865349+00:00 bl-cassoop-p11 kernel: [74739.881595]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.865350+00:00 bl-cassoop-p11 kernel: [74739.881598]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:49:27.865354+00:00 bl-cassoop-p11 kernel: [74739.881600]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:49:27.865356+00:00 bl-cassoop-p11 kernel: [74739.881603]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:49:27.865357+00:00 bl-cassoop-p11 kernel: [74739.881605]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:49:27.865359+00:00 bl-cassoop-p11 kernel: [74739.881607]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:49:27.865360+00:00 bl-cassoop-p11 kernel: [74739.881610]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:49:27.865362+00:00 bl-cassoop-p11 kernel: [74739.881612]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:49:27.865363+00:00 bl-cassoop-p11 kernel: [74739.881615]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:49:27.865365+00:00 bl-cassoop-p11 kernel: [74739.881618]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:49:27.865369+00:00 bl-cassoop-p11 kernel: [74739.881620]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:49:27.865370+00:00 bl-cassoop-p11 kernel: [74739.881623]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:49:27.865372+00:00 bl-cassoop-p11 kernel: [74739.881626]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:49:27.865373+00:00 bl-cassoop-p11 kernel: [74739.881628]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:49:27.865375+00:00 bl-cassoop-p11 kernel: [74739.881630]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:49:27.865377+00:00 bl-cassoop-p11 kernel: [74739.881633]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:49:27.865378+00:00 bl-cassoop-p11 kernel: [74739.881635]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:49:27.865382+00:00 bl-cassoop-p11 kernel: [74739.881637]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:49:27.865384+00:00 bl-cassoop-p11 kernel: [74739.881640]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:49:27.865385+00:00 bl-cassoop-p11 kernel: [74739.881642]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:49:27.865387+00:00 bl-cassoop-p11 kernel: [74739.881645]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:49:27.865388+00:00 bl-cassoop-p11 kernel: [74739.881648]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:49:27.865454+00:00 bl-cassoop-p11 kernel: [74739.881650]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:49:27.865456+00:00 bl-cassoop-p11 kernel: [74739.881652]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:49:27.865457+00:00 bl-cassoop-p11 kernel: [74739.881654]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.865461+00:00 bl-cassoop-p11 kernel: [74739.881657]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865462+00:00 bl-cassoop-p11 kernel: [74739.881660]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.865464+00:00 bl-cassoop-p11 kernel: [74739.881662]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865466+00:00 bl-cassoop-p11 kernel: [74739.881681] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.865468+00:00 bl-cassoop-p11 kernel: [74739.881683] NMI backtrace for cpu 15
2014-01-18T13:49:27.865469+00:00 bl-cassoop-p11 kernel: [74739.881685] CPU: 15 PID: 36402 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865471+00:00 bl-cassoop-p11 kernel: [74739.881686] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865474+00:00 bl-cassoop-p11 kernel: [74739.881687] task: ffff8818979f5dc0 ti: ffff88252a4ca000 task.ti: ffff88252a4ca000
2014-01-18T13:49:27.865476+00:00 bl-cassoop-p11 kernel: [74739.881690] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.865478+00:00 bl-cassoop-p11 kernel: [74739.881690] RSP: 0018:ffff88252a4cbc88  EFLAGS: 00000087
2014-01-18T13:49:27.865479+00:00 bl-cassoop-p11 kernel: [74739.881691] RAX: ffff88252a4cbcc0 RBX: ffff8818979f5dc0 RCX: 0000000000000001
2014-01-18T13:49:27.865481+00:00 bl-cassoop-p11 kernel: [74739.881692] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.865482+00:00 bl-cassoop-p11 kernel: [74739.881692] RBP: ffff88252a4cbc88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.865484+00:00 bl-cassoop-p11 kernel: [74739.881693] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:27.865487+00:00 bl-cassoop-p11 kernel: [74739.881693] R13: ffff8818979f5dc0 R14: 0000000000000000 R15: 0000000000000002
2014-01-18T13:49:27.865489+00:00 bl-cassoop-p11 kernel: [74739.881694] FS:  0000000000000000(0000) GS:ffff88301f2e0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865490+00:00 bl-cassoop-p11 kernel: [74739.881695] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865491+00:00 bl-cassoop-p11 kernel: [74739.881696] CR2: 00000006d89af728 CR3: 0000001885e2a000 CR4: 00000000000407e0
2014-01-18T13:49:27.865492+00:00 bl-cassoop-p11 kernel: [74739.881696] Stack:
2014-01-18T13:49:27.865494+00:00 bl-cassoop-p11 kernel: [74739.881699]  ffff88252a4cbc98 ffffffff8174787e ffff88252a4cbd08 ffffffff810676e5
2014-01-18T13:49:27.865495+00:00 bl-cassoop-p11 kernel: [74739.881701]  ffff88252a4cbd28 ffff88252a4cbcc0 ffff8818979f5dc0 ffff88252a4cbcc0
2014-01-18T13:49:27.865497+00:00 bl-cassoop-p11 kernel: [74739.881703]  ffff88252a4cbcc0 0000000100000000 0001d7ce00000001 ffff8818979f5dc0
2014-01-18T13:49:27.865499+00:00 bl-cassoop-p11 kernel: [74739.881703] Call Trace:
2014-01-18T13:49:27.865501+00:00 bl-cassoop-p11 kernel: [74739.881706]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.865503+00:00 bl-cassoop-p11 kernel: [74739.881708]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.865504+00:00 bl-cassoop-p11 kernel: [74739.881710]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.865506+00:00 bl-cassoop-p11 kernel: [74739.881712]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.865507+00:00 bl-cassoop-p11 kernel: [74739.881715]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:27.865509+00:00 bl-cassoop-p11 kernel: [74739.881716]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:27.865512+00:00 bl-cassoop-p11 kernel: [74739.881718]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:27.865514+00:00 bl-cassoop-p11 kernel: [74739.881721]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:27.865515+00:00 bl-cassoop-p11 kernel: [74739.881722]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:27.865517+00:00 bl-cassoop-p11 kernel: [74739.881725]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:27.865518+00:00 bl-cassoop-p11 kernel: [74739.881727]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:27.865520+00:00 bl-cassoop-p11 kernel: [74739.881729]  [<ffffffff81747c7c>] retint_signal+0x48/0x8c
2014-01-18T13:49:27.865522+00:00 bl-cassoop-p11 kernel: [74739.881741] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.865526+00:00 bl-cassoop-p11 kernel: [74739.881742] NMI backtrace for cpu 17
2014-01-18T13:49:27.865527+00:00 bl-cassoop-p11 kernel: [74739.881744] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865529+00:00 bl-cassoop-p11 kernel: [74739.881745] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865530+00:00 bl-cassoop-p11 kernel: [74739.881746] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:49:27.865532+00:00 bl-cassoop-p11 kernel: [74739.881748] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.865534+00:00 bl-cassoop-p11 kernel: [74739.881749] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:49:27.865535+00:00 bl-cassoop-p11 kernel: [74739.881750] RAX: 0000000000000006 RBX: ffff88301f315088 RCX: ffff88181fa78748
2014-01-18T13:49:27.865538+00:00 bl-cassoop-p11 kernel: [74739.881751] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.865540+00:00 bl-cassoop-p11 kernel: [74739.881752] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:49:27.865541+00:00 bl-cassoop-p11 kernel: [74739.881752] R10: 0000000000000004 R11: 0000000000017960 R12: ffff88301f315080
2014-01-18T13:49:27.865542+00:00 bl-cassoop-p11 kernel: [74739.881753] R13: ffffffff8105ae90 R14: ffff882feccbfa10 R15: 0000000000015040
2014-01-18T13:49:27.865653+00:00 bl-cassoop-p11 kernel: [74739.881755] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865654+00:00 bl-cassoop-p11 kernel: [74739.881755] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865656+00:00 bl-cassoop-p11 kernel: [74739.881756] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.865657+00:00 bl-cassoop-p11 kernel: [74739.881757] Stack:
2014-01-18T13:49:27.865661+00:00 bl-cassoop-p11 kernel: [74739.881760]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:49:27.865663+00:00 bl-cassoop-p11 kernel: [74739.881763]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:49:27.865664+00:00 bl-cassoop-p11 kernel: [74739.881766]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:49:27.865665+00:00 bl-cassoop-p11 kernel: [74739.881767] Call Trace:
2014-01-18T13:49:27.865667+00:00 bl-cassoop-p11 kernel: [74739.881770]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.865668+00:00 bl-cassoop-p11 kernel: [74739.881772]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.865670+00:00 bl-cassoop-p11 kernel: [74739.881774]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.865671+00:00 bl-cassoop-p11 kernel: [74739.881777]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:27.865675+00:00 bl-cassoop-p11 kernel: [74739.881779]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:27.865677+00:00 bl-cassoop-p11 kernel: [74739.881781]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:27.865678+00:00 bl-cassoop-p11 kernel: [74739.881784]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:27.865680+00:00 bl-cassoop-p11 kernel: [74739.881786]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:27.865681+00:00 bl-cassoop-p11 kernel: [74739.881788]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:27.865683+00:00 bl-cassoop-p11 kernel: [74739.881790]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:27.865685+00:00 bl-cassoop-p11 kernel: [74739.881803]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:27.865688+00:00 bl-cassoop-p11 kernel: [74739.881805]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:27.865690+00:00 bl-cassoop-p11 kernel: [74739.881807]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:27.865691+00:00 bl-cassoop-p11 kernel: [74739.881810]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:27.865693+00:00 bl-cassoop-p11 kernel: [74739.881812]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.865694+00:00 bl-cassoop-p11 kernel: [74739.881815]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.865696+00:00 bl-cassoop-p11 kernel: [74739.881817]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:49:27.865697+00:00 bl-cassoop-p11 kernel: [74739.881819]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:49:27.865742+00:00 bl-cassoop-p11 kernel: [74739.881822]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:49:27.865746+00:00 bl-cassoop-p11 kernel: [74739.881824]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:49:27.865748+00:00 bl-cassoop-p11 kernel: [74739.881827]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:49:27.865749+00:00 bl-cassoop-p11 kernel: [74739.881829]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.865751+00:00 bl-cassoop-p11 kernel: [74739.881832]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.865753+00:00 bl-cassoop-p11 kernel: [74739.881851] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.865755+00:00 bl-cassoop-p11 kernel: [74739.881852] NMI backtrace for cpu 18
2014-01-18T13:49:27.865756+00:00 bl-cassoop-p11 kernel: [74739.881854] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865760+00:00 bl-cassoop-p11 kernel: [74739.881855] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865762+00:00 bl-cassoop-p11 kernel: [74739.881856] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:49:27.865764+00:00 bl-cassoop-p11 kernel: [74739.881858] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:27.865765+00:00 bl-cassoop-p11 kernel: [74739.881859] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:49:27.865766+00:00 bl-cassoop-p11 kernel: [74739.881860] RAX: 0000000000000282 RBX: ffff882ff8361750 RCX: 0000000000000001
2014-01-18T13:49:27.865768+00:00 bl-cassoop-p11 kernel: [74739.881861] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:49:27.865769+00:00 bl-cassoop-p11 kernel: [74739.881862] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:49:27.865772+00:00 bl-cassoop-p11 kernel: [74739.881863] R10: 0000000000000001 R11: 0000000000000400 R12: 0000000000000002
2014-01-18T13:49:27.865774+00:00 bl-cassoop-p11 kernel: [74739.881864] R13: ffff88181fa35040 R14: ffff88181fa35050 R15: 0000000000000001
2014-01-18T13:49:27.865775+00:00 bl-cassoop-p11 kernel: [74739.881865] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865777+00:00 bl-cassoop-p11 kernel: [74739.881866] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865778+00:00 bl-cassoop-p11 kernel: [74739.881867] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.865779+00:00 bl-cassoop-p11 kernel: [74739.881867] Stack:
2014-01-18T13:49:27.865781+00:00 bl-cassoop-p11 kernel: [74739.881871]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:49:27.865782+00:00 bl-cassoop-p11 kernel: [74739.881874]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:49:27.865786+00:00 bl-cassoop-p11 kernel: [74739.881877]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:49:27.865787+00:00 bl-cassoop-p11 kernel: [74739.881878] Call Trace:
2014-01-18T13:49:27.865788+00:00 bl-cassoop-p11 kernel: [74739.881881]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.865790+00:00 bl-cassoop-p11 kernel: [74739.881883]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.865792+00:00 bl-cassoop-p11 kernel: [74739.881885]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.865793+00:00 bl-cassoop-p11 kernel: [74739.881887]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.865865+00:00 bl-cassoop-p11 kernel: [74739.881890]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.865870+00:00 bl-cassoop-p11 kernel: [74739.881892]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.865872+00:00 bl-cassoop-p11 kernel: [74739.881895]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.865873+00:00 bl-cassoop-p11 kernel: [74739.881898]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.865875+00:00 bl-cassoop-p11 kernel: [74739.881900]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:27.865876+00:00 bl-cassoop-p11 kernel: [74739.881903]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:27.865878+00:00 bl-cassoop-p11 kernel: [74739.881905]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:49:27.865880+00:00 bl-cassoop-p11 kernel: [74739.881907]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:27.865881+00:00 bl-cassoop-p11 kernel: [74739.881910]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:27.865885+00:00 bl-cassoop-p11 kernel: [74739.881913]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:27.865886+00:00 bl-cassoop-p11 kernel: [74739.881915]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:27.865888+00:00 bl-cassoop-p11 kernel: [74739.881917]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:49:27.865889+00:00 bl-cassoop-p11 kernel: [74739.881920]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.865891+00:00 bl-cassoop-p11 kernel: [74739.881923]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.865893+00:00 bl-cassoop-p11 kernel: [74739.881926]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.865894+00:00 bl-cassoop-p11 kernel: [74739.881928]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.865898+00:00 bl-cassoop-p11 kernel: [74739.881931]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.865899+00:00 bl-cassoop-p11 kernel: [74739.881933]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:27.865901+00:00 bl-cassoop-p11 kernel: [74739.881936]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:27.865902+00:00 bl-cassoop-p11 kernel: [74739.881938]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:27.865904+00:00 bl-cassoop-p11 kernel: [74739.881941]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:27.865905+00:00 bl-cassoop-p11 kernel: [74739.881943]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:27.865907+00:00 bl-cassoop-p11 kernel: [74739.881946]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.865908+00:00 bl-cassoop-p11 kernel: [74739.881949]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865911+00:00 bl-cassoop-p11 kernel: [74739.881951]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.865913+00:00 bl-cassoop-p11 kernel: [74739.881954]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.865915+00:00 bl-cassoop-p11 kernel: [74739.881972] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:27.865916+00:00 bl-cassoop-p11 kernel: [74739.881974] NMI backtrace for cpu 20
2014-01-18T13:49:27.865918+00:00 bl-cassoop-p11 kernel: [74739.881975] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.865920+00:00 bl-cassoop-p11 kernel: [74739.881976] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.865921+00:00 bl-cassoop-p11 kernel: [74739.881976] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:49:27.865925+00:00 bl-cassoop-p11 kernel: [74739.881978] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:27.865927+00:00 bl-cassoop-p11 kernel: [74739.881979] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:49:27.865928+00:00 bl-cassoop-p11 kernel: [74739.881980] RAX: 0000000000000002 RBX: ffff88181fb55088 RCX: ffff88181fa387c0
2014-01-18T13:49:27.865930+00:00 bl-cassoop-p11 kernel: [74739.881980] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.865931+00:00 bl-cassoop-p11 kernel: [74739.881981] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:49:27.865933+00:00 bl-cassoop-p11 kernel: [74739.881981] R10: 0000000000000004 R11: 0000000000000206 R12: ffff88181fb55080
2014-01-18T13:49:27.865934+00:00 bl-cassoop-p11 kernel: [74739.881982] R13: ffffffff8105ae90 R14: ffff8801111b3d50 R15: 0000000000015040
2014-01-18T13:49:27.865937+00:00 bl-cassoop-p11 kernel: [74739.881983] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.865939+00:00 bl-cassoop-p11 kernel: [74739.881984] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.865940+00:00 bl-cassoop-p11 kernel: [74739.881984] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.865941+00:00 bl-cassoop-p11 kernel: [74739.881984] Stack:
2014-01-18T13:49:27.865943+00:00 bl-cassoop-p11 kernel: [74739.881987]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:49:27.865944+00:00 bl-cassoop-p11 kernel: [74739.881989]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:49:27.865946+00:00 bl-cassoop-p11 kernel: [74739.881991]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:49:27.865952+00:00 bl-cassoop-p11 kernel: [74739.881991] Call Trace:
2014-01-18T13:49:27.865954+00:00 bl-cassoop-p11 kernel: [74739.881994]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.865956+00:00 bl-cassoop-p11 kernel: [74739.881996]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.865957+00:00 bl-cassoop-p11 kernel: [74739.881998]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:27.865959+00:00 bl-cassoop-p11 kernel: [74739.882000]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:27.865960+00:00 bl-cassoop-p11 kernel: [74739.882001]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:27.865962+00:00 bl-cassoop-p11 kernel: [74739.882003]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:27.865964+00:00 bl-cassoop-p11 kernel: [74739.882005]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:27.865968+00:00 bl-cassoop-p11 kernel: [74739.882007]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:27.866075+00:00 bl-cassoop-p11 kernel: [74739.882009]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.866078+00:00 bl-cassoop-p11 kernel: [74739.882020] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:27.866079+00:00 bl-cassoop-p11 kernel: [74739.882021] NMI backtrace for cpu 19
2014-01-18T13:49:27.866081+00:00 bl-cassoop-p11 kernel: [74739.882024] CPU: 19 PID: 0 Comm: swapper/19 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866083+00:00 bl-cassoop-p11 kernel: [74739.882025] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866084+00:00 bl-cassoop-p11 kernel: [74739.882026] task: ffff8817fadb8000 ti: ffff8817fadb6000 task.ti: ffff8817fadb6000
2014-01-18T13:49:27.866088+00:00 bl-cassoop-p11 kernel: [74739.882029] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866090+00:00 bl-cassoop-p11 kernel: [74739.882030] RSP: 0018:ffff8817fadb7da8  EFLAGS: 00000046
2014-01-18T13:49:27.866091+00:00 bl-cassoop-p11 kernel: [74739.882031] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866093+00:00 bl-cassoop-p11 kernel: [74739.882032] RDX: 0000000000000000 RSI: ffff8817fadb7fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.866094+00:00 bl-cassoop-p11 kernel: [74739.882033] RBP: ffff8817fadb7dd8 R08: 0000000000009c61 R09: 000000002f063440
2014-01-18T13:49:27.866095+00:00 bl-cassoop-p11 kernel: [74739.882034] R10: 00000000000017f6 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866097+00:00 bl-cassoop-p11 kernel: [74739.882034] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866100+00:00 bl-cassoop-p11 kernel: [74739.882036] FS:  0000000000000000(0000) GS:ffff88301f320000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866102+00:00 bl-cassoop-p11 kernel: [74739.882037] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866103+00:00 bl-cassoop-p11 kernel: [74739.882037] CR2: 00000006dc018550 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866104+00:00 bl-cassoop-p11 kernel: [74739.882038] Stack:
2014-01-18T13:49:27.866105+00:00 bl-cassoop-p11 kernel: [74739.882042]  ffff8817fadb7dd8 00000013810bc824 ffffe8ffffb20320 000043f35e6315e8
2014-01-18T13:49:27.866107+00:00 bl-cassoop-p11 kernel: [74739.882045]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadb7e38 ffffffff815e5a9f
2014-01-18T13:49:27.866108+00:00 bl-cassoop-p11 kernel: [74739.882049]  0000000000000000 0000000009cb9094 0000000000000000 0000000009cb9094
2014-01-18T13:49:27.866109+00:00 bl-cassoop-p11 kernel: [74739.882049] Call Trace:
2014-01-18T13:49:27.866113+00:00 bl-cassoop-p11 kernel: [74739.882052]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866115+00:00 bl-cassoop-p11 kernel: [74739.882054]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866116+00:00 bl-cassoop-p11 kernel: [74739.882057]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866118+00:00 bl-cassoop-p11 kernel: [74739.882059]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866119+00:00 bl-cassoop-p11 kernel: [74739.882062]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866121+00:00 bl-cassoop-p11 kernel: [74739.882065]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866123+00:00 bl-cassoop-p11 kernel: [74739.882084] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866127+00:00 bl-cassoop-p11 kernel: [74739.882085] NMI backtrace for cpu 22
2014-01-18T13:49:27.866129+00:00 bl-cassoop-p11 kernel: [74739.882087] CPU: 22 PID: 0 Comm: swapper/22 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866130+00:00 bl-cassoop-p11 kernel: [74739.882088] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866132+00:00 bl-cassoop-p11 kernel: [74739.882089] task: ffff8817fadbc650 ti: ffff8817fadc4000 task.ti: ffff8817fadc4000
2014-01-18T13:49:27.866134+00:00 bl-cassoop-p11 kernel: [74739.882092] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866135+00:00 bl-cassoop-p11 kernel: [74739.882093] RSP: 0018:ffff8817fadc5da8  EFLAGS: 00000046
2014-01-18T13:49:27.866136+00:00 bl-cassoop-p11 kernel: [74739.882094] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866140+00:00 bl-cassoop-p11 kernel: [74739.882095] RDX: 0000000000000000 RSI: ffff8817fadc5fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.866141+00:00 bl-cassoop-p11 kernel: [74739.882096] RBP: ffff8817fadc5dd8 R08: 0000000000000ee5 R09: 00000000009d2151
2014-01-18T13:49:27.866143+00:00 bl-cassoop-p11 kernel: [74739.882096] R10: 0000000000001dcd R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866144+00:00 bl-cassoop-p11 kernel: [74739.882097] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866145+00:00 bl-cassoop-p11 kernel: [74739.882099] FS:  0000000000000000(0000) GS:ffff88181fb60000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866147+00:00 bl-cassoop-p11 kernel: [74739.882099] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866148+00:00 bl-cassoop-p11 kernel: [74739.882100] CR2: 000000008b06d0b0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866149+00:00 bl-cassoop-p11 kernel: [74739.882101] Stack:
2014-01-18T13:49:27.866152+00:00 bl-cassoop-p11 kernel: [74739.882105]  ffff8817fadc5dd8 00000016810bc824 ffffe8e800360320 000043f364aef59f
2014-01-18T13:49:27.866154+00:00 bl-cassoop-p11 kernel: [74739.882108]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc5e38 ffffffff815e5a9f
2014-01-18T13:49:27.866156+00:00 bl-cassoop-p11 kernel: [74739.882111]  0000000000000000 00000000009908f2 0000000000000000 00000000009908f2
2014-01-18T13:49:27.866157+00:00 bl-cassoop-p11 kernel: [74739.882112] Call Trace:
2014-01-18T13:49:27.866159+00:00 bl-cassoop-p11 kernel: [74739.882114]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866161+00:00 bl-cassoop-p11 kernel: [74739.882117]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866162+00:00 bl-cassoop-p11 kernel: [74739.882119]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866224+00:00 bl-cassoop-p11 kernel: [74739.882122]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866226+00:00 bl-cassoop-p11 kernel: [74739.882125]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866228+00:00 bl-cassoop-p11 kernel: [74739.882127]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866231+00:00 bl-cassoop-p11 kernel: [74739.882146] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866232+00:00 bl-cassoop-p11 kernel: [74739.882148] NMI backtrace for cpu 21
2014-01-18T13:49:27.866234+00:00 bl-cassoop-p11 kernel: [74739.882150] CPU: 21 PID: 0 Comm: swapper/21 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866236+00:00 bl-cassoop-p11 kernel: [74739.882151] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866240+00:00 bl-cassoop-p11 kernel: [74739.882152] task: ffff8817fadbaee0 ti: ffff8817fadc2000 task.ti: ffff8817fadc2000
2014-01-18T13:49:27.866242+00:00 bl-cassoop-p11 kernel: [74739.882156] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866244+00:00 bl-cassoop-p11 kernel: [74739.882157] RSP: 0018:ffff8817fadc3da8  EFLAGS: 00000046
2014-01-18T13:49:27.866245+00:00 bl-cassoop-p11 kernel: [74739.882157] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866247+00:00 bl-cassoop-p11 kernel: [74739.882158] RDX: 0000000000000000 RSI: ffff8817fadc3fd8 RDI: 0000000000000015
2014-01-18T13:49:27.866248+00:00 bl-cassoop-p11 kernel: [74739.882159] RBP: ffff8817fadc3dd8 R08: 000000000001f255 R09: 000000014a354a81
2014-01-18T13:49:27.866250+00:00 bl-cassoop-p11 kernel: [74739.882160] R10: 00000000000019d2 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866252+00:00 bl-cassoop-p11 kernel: [74739.882160] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866255+00:00 bl-cassoop-p11 kernel: [74739.882161] FS:  0000000000000000(0000) GS:ffff88301f340000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866257+00:00 bl-cassoop-p11 kernel: [74739.882162] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866258+00:00 bl-cassoop-p11 kernel: [74739.882163] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866260+00:00 bl-cassoop-p11 kernel: [74739.882163] Stack:
2014-01-18T13:49:27.866261+00:00 bl-cassoop-p11 kernel: [74739.882166]  ffff8817fadc3dd8 00000015810bc824 ffffe8ffffb40320 000043f3631e3443
2014-01-18T13:49:27.866263+00:00 bl-cassoop-p11 kernel: [74739.882169]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc3e38 ffffffff815e5a9f
2014-01-18T13:49:27.866264+00:00 bl-cassoop-p11 kernel: [74739.882171]  0000000000000000 0000000015e7be04 0000000000000000 0000000015e7be04
2014-01-18T13:49:27.866267+00:00 bl-cassoop-p11 kernel: [74739.882171] Call Trace:
2014-01-18T13:49:27.866269+00:00 bl-cassoop-p11 kernel: [74739.882174]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866271+00:00 bl-cassoop-p11 kernel: [74739.882175]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866273+00:00 bl-cassoop-p11 kernel: [74739.882177]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866275+00:00 bl-cassoop-p11 kernel: [74739.882179]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866276+00:00 bl-cassoop-p11 kernel: [74739.882181]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866278+00:00 bl-cassoop-p11 kernel: [74739.882184]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866281+00:00 bl-cassoop-p11 kernel: [74739.882196] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866284+00:00 bl-cassoop-p11 kernel: [74739.882197] NMI backtrace for cpu 24
2014-01-18T13:49:27.866286+00:00 bl-cassoop-p11 kernel: [74739.882199] CPU: 24 PID: 0 Comm: swapper/24 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866288+00:00 bl-cassoop-p11 kernel: [74739.882200] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866290+00:00 bl-cassoop-p11 kernel: [74739.882201] task: ffff8817fadc8000 ti: ffff8817fadd0000 task.ti: ffff8817fadd0000
2014-01-18T13:49:27.866292+00:00 bl-cassoop-p11 kernel: [74739.882204] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866294+00:00 bl-cassoop-p11 kernel: [74739.882205] RSP: 0018:ffff8817fadd1da8  EFLAGS: 00000046
2014-01-18T13:49:27.866295+00:00 bl-cassoop-p11 kernel: [74739.882206] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866299+00:00 bl-cassoop-p11 kernel: [74739.882207] RDX: 0000000000000000 RSI: ffff8817fadd1fd8 RDI: 0000000000000018
2014-01-18T13:49:27.866301+00:00 bl-cassoop-p11 kernel: [74739.882207] RBP: ffff8817fadd1dd8 R08: 0000000000026f51 R09: 00000000275fd8a4
2014-01-18T13:49:27.866302+00:00 bl-cassoop-p11 kernel: [74739.882208] R10: 0000000000001fcb R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866303+00:00 bl-cassoop-p11 kernel: [74739.882209] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866305+00:00 bl-cassoop-p11 kernel: [74739.882210] FS:  0000000000000000(0000) GS:ffff88181fb80000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866306+00:00 bl-cassoop-p11 kernel: [74739.882211] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866308+00:00 bl-cassoop-p11 kernel: [74739.882212] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866311+00:00 bl-cassoop-p11 kernel: [74739.882213] Stack:
2014-01-18T13:49:27.866313+00:00 bl-cassoop-p11 kernel: [74739.882217]  ffff8817fadd1dd8 00000018810bc824 ffffe8e800380320 000043f35cbd7ae4
2014-01-18T13:49:27.866314+00:00 bl-cassoop-p11 kernel: [74739.882220]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd1e38 ffffffff815e5a9f
2014-01-18T13:49:27.866316+00:00 bl-cassoop-p11 kernel: [74739.882223]  0000000000000000 000000001c9ca8e4 0000000000000000 000000001c9ca8e4
2014-01-18T13:49:27.866317+00:00 bl-cassoop-p11 kernel: [74739.882224] Call Trace:
2014-01-18T13:49:27.866319+00:00 bl-cassoop-p11 kernel: [74739.882226]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866413+00:00 bl-cassoop-p11 kernel: [74739.882228]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866415+00:00 bl-cassoop-p11 kernel: [74739.882231]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866420+00:00 bl-cassoop-p11 kernel: [74739.882234]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866422+00:00 bl-cassoop-p11 kernel: [74739.882236]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866423+00:00 bl-cassoop-p11 kernel: [74739.882239]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866426+00:00 bl-cassoop-p11 kernel: [74739.882258] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866427+00:00 bl-cassoop-p11 kernel: [74739.882260] NMI backtrace for cpu 23
2014-01-18T13:49:27.866429+00:00 bl-cassoop-p11 kernel: [74739.882262] CPU: 23 PID: 0 Comm: swapper/23 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866430+00:00 bl-cassoop-p11 kernel: [74739.882263] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866434+00:00 bl-cassoop-p11 kernel: [74739.882264] task: ffff8817fadbddc0 ti: ffff8817fadc6000 task.ti: ffff8817fadc6000
2014-01-18T13:49:27.866436+00:00 bl-cassoop-p11 kernel: [74739.882267] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866437+00:00 bl-cassoop-p11 kernel: [74739.882268] RSP: 0018:ffff8817fadc7da8  EFLAGS: 00000046
2014-01-18T13:49:27.866439+00:00 bl-cassoop-p11 kernel: [74739.882269] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866440+00:00 bl-cassoop-p11 kernel: [74739.882269] RDX: 0000000000000000 RSI: ffff8817fadc7fd8 RDI: 0000000000000017
2014-01-18T13:49:27.866442+00:00 bl-cassoop-p11 kernel: [74739.882270] RBP: ffff8817fadc7dd8 R08: 0000000000016e39 R09: 00000001e8147119
2014-01-18T13:49:27.866443+00:00 bl-cassoop-p11 kernel: [74739.882271] R10: 00000000000016cd R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866446+00:00 bl-cassoop-p11 kernel: [74739.882271] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866448+00:00 bl-cassoop-p11 kernel: [74739.882272] FS:  0000000000000000(0000) GS:ffff88301f360000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866450+00:00 bl-cassoop-p11 kernel: [74739.882273] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866451+00:00 bl-cassoop-p11 kernel: [74739.882274] CR2: 00007fe8cd9a3000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866452+00:00 bl-cassoop-p11 kernel: [74739.882274] Stack:
2014-01-18T13:49:27.866453+00:00 bl-cassoop-p11 kernel: [74739.882277]  ffff8817fadc7dd8 00000017810bc824 ffffe8ffffb60320 000043f35c06ade5
2014-01-18T13:49:27.866455+00:00 bl-cassoop-p11 kernel: [74739.882279]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc7e38 ffffffff815e5a9f
2014-01-18T13:49:27.866456+00:00 bl-cassoop-p11 kernel: [74739.882281]  0000000000000000 00000000118aab15 0000000000000000 00000000118aab15
2014-01-18T13:49:27.866459+00:00 bl-cassoop-p11 kernel: [74739.882281] Call Trace:
2014-01-18T13:49:27.866461+00:00 bl-cassoop-p11 kernel: [74739.882284]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866462+00:00 bl-cassoop-p11 kernel: [74739.882286]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866464+00:00 bl-cassoop-p11 kernel: [74739.882288]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866466+00:00 bl-cassoop-p11 kernel: [74739.882290]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866467+00:00 bl-cassoop-p11 kernel: [74739.882292]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866469+00:00 bl-cassoop-p11 kernel: [74739.882294]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866473+00:00 bl-cassoop-p11 kernel: [74739.882306] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866474+00:00 bl-cassoop-p11 kernel: [74739.882308] NMI backtrace for cpu 25
2014-01-18T13:49:27.866476+00:00 bl-cassoop-p11 kernel: [74739.882310] CPU: 25 PID: 0 Comm: swapper/25 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866478+00:00 bl-cassoop-p11 kernel: [74739.882311] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866479+00:00 bl-cassoop-p11 kernel: [74739.882312] task: ffff8817fadc9770 ti: ffff8817fadd2000 task.ti: ffff8817fadd2000
2014-01-18T13:49:27.866481+00:00 bl-cassoop-p11 kernel: [74739.882315] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866483+00:00 bl-cassoop-p11 kernel: [74739.882315] RSP: 0018:ffff8817fadd3da8  EFLAGS: 00000046
2014-01-18T13:49:27.866486+00:00 bl-cassoop-p11 kernel: [74739.882316] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866487+00:00 bl-cassoop-p11 kernel: [74739.882317] RDX: 0000000000000000 RSI: ffff8817fadd3fd8 RDI: 0000000000000019
2014-01-18T13:49:27.866489+00:00 bl-cassoop-p11 kernel: [74739.882317] RBP: ffff8817fadd3dd8 R08: 000000000000bc19 R09: 00000000000348ea
2014-01-18T13:49:27.866490+00:00 bl-cassoop-p11 kernel: [74739.882318] R10: 0000000000001b37 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866492+00:00 bl-cassoop-p11 kernel: [74739.882318] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866493+00:00 bl-cassoop-p11 kernel: [74739.882319] FS:  0000000000000000(0000) GS:ffff88301f380000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866495+00:00 bl-cassoop-p11 kernel: [74739.882320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866496+00:00 bl-cassoop-p11 kernel: [74739.882320] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866499+00:00 bl-cassoop-p11 kernel: [74739.882321] Stack:
2014-01-18T13:49:27.866501+00:00 bl-cassoop-p11 kernel: [74739.882323]  ffff8817fadd3dd8 00000019810bc824 ffffe8ffffb80320 000043f35d749906
2014-01-18T13:49:27.866502+00:00 bl-cassoop-p11 kernel: [74739.882325]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd3e38 ffffffff815e5a9f
2014-01-18T13:49:27.866504+00:00 bl-cassoop-p11 kernel: [74739.882328]  0000000000000000 000000001b667901 0000000000000000 000000001b667901
2014-01-18T13:49:27.866571+00:00 bl-cassoop-p11 kernel: [74739.882328] Call Trace:
2014-01-18T13:49:27.866573+00:00 bl-cassoop-p11 kernel: [74739.882330]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866574+00:00 bl-cassoop-p11 kernel: [74739.882332]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866582+00:00 bl-cassoop-p11 kernel: [74739.882334]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866584+00:00 bl-cassoop-p11 kernel: [74739.882347]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866585+00:00 bl-cassoop-p11 kernel: [74739.882349]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866587+00:00 bl-cassoop-p11 kernel: [74739.882351]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866589+00:00 bl-cassoop-p11 kernel: [74739.882362] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866591+00:00 bl-cassoop-p11 kernel: [74739.882366] NMI backtrace for cpu 26
2014-01-18T13:49:27.866592+00:00 bl-cassoop-p11 kernel: [74739.882368] CPU: 26 PID: 0 Comm: swapper/26 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866596+00:00 bl-cassoop-p11 kernel: [74739.882372] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866597+00:00 bl-cassoop-p11 kernel: [74739.882382] task: ffff8817fadcaee0 ti: ffff8817fadd4000 task.ti: ffff8817fadd4000
2014-01-18T13:49:27.866599+00:00 bl-cassoop-p11 kernel: [74739.882391] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.866600+00:00 bl-cassoop-p11 kernel: [74739.882393] RSP: 0018:ffff8817fadd5da8  EFLAGS: 00000046
2014-01-18T13:49:27.866602+00:00 bl-cassoop-p11 kernel: [74739.882395] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.866604+00:00 bl-cassoop-p11 kernel: [74739.882396] RDX: 0000000000000000 RSI: ffff8817fadd5fd8 RDI: 000000000000001a
2014-01-18T13:49:27.866605+00:00 bl-cassoop-p11 kernel: [74739.882396] RBP: ffff8817fadd5dd8 R08: 000000000001cde5 R09: 000000007ee5d689
2014-01-18T13:49:27.866606+00:00 bl-cassoop-p11 kernel: [74739.882397] R10: 0000000000001a8e R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.866610+00:00 bl-cassoop-p11 kernel: [74739.882397] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.866611+00:00 bl-cassoop-p11 kernel: [74739.882398] FS:  0000000000000000(0000) GS:ffff88181fba0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866613+00:00 bl-cassoop-p11 kernel: [74739.882399] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866614+00:00 bl-cassoop-p11 kernel: [74739.882400] CR2: 000000008b06c008 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.866615+00:00 bl-cassoop-p11 kernel: [74739.882402] Stack:
2014-01-18T13:49:27.866617+00:00 bl-cassoop-p11 kernel: [74739.882423]  ffff8817fadd5dd8 0000001a810bc824 ffffe8e8003a0320 000043f35e2bb651
2014-01-18T13:49:27.866618+00:00 bl-cassoop-p11 kernel: [74739.882429]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd5e38 ffffffff815e5a9f
2014-01-18T13:49:27.866621+00:00 bl-cassoop-p11 kernel: [74739.882436]  0000000000000000 00000000160dbd12 0000000000000000 00000000160dbd12
2014-01-18T13:49:27.866622+00:00 bl-cassoop-p11 kernel: [74739.882438] Call Trace:
2014-01-18T13:49:27.866624+00:00 bl-cassoop-p11 kernel: [74739.882448]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.866626+00:00 bl-cassoop-p11 kernel: [74739.882455]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.866627+00:00 bl-cassoop-p11 kernel: [74739.882461]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.866629+00:00 bl-cassoop-p11 kernel: [74739.882474]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.866630+00:00 bl-cassoop-p11 kernel: [74739.882476]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.866632+00:00 bl-cassoop-p11 kernel: [74739.882493]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.866636+00:00 bl-cassoop-p11 kernel: [74739.882511] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.866638+00:00 bl-cassoop-p11 kernel: [74739.882513] NMI backtrace for cpu 27
2014-01-18T13:49:27.866639+00:00 bl-cassoop-p11 kernel: [74739.882515] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.866641+00:00 bl-cassoop-p11 kernel: [74739.882515] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.866643+00:00 bl-cassoop-p11 kernel: [74739.882516] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:49:27.866645+00:00 bl-cassoop-p11 kernel: [74739.882518] RIP: 0010:[<ffffffff8101b436>]  [<ffffffff8101b436>] native_read_tsc+0x6/0x20
2014-01-18T13:49:27.866646+00:00 bl-cassoop-p11 kernel: [74739.882519] RSP: 0018:ffff88301f3a3c68  EFLAGS: 00000006
2014-01-18T13:49:27.866650+00:00 bl-cassoop-p11 kernel: [74739.882520] RAX: 000000003aec2a7b RBX: 00000000000003e9 RCX: 000000003aec2a56
2014-01-18T13:49:27.866651+00:00 bl-cassoop-p11 kernel: [74739.882520] RDX: 000000000000b118 RSI: 0000000000000100 RDI: 000000000003efca
2014-01-18T13:49:27.866653+00:00 bl-cassoop-p11 kernel: [74739.882521] RBP: ffff88301f3a3c68 R08: ffffffff81d03da0 R09: 0000000000000000
2014-01-18T13:49:27.866654+00:00 bl-cassoop-p11 kernel: [74739.882521] R10: 0000000000000038 R11: 000000000001ca98 R12: 000000003aec2a56
2014-01-18T13:49:27.866655+00:00 bl-cassoop-p11 kernel: [74739.882522] R13: 000000000000001b R14: 000000000003efca R15: 0000000000000096
2014-01-18T13:49:27.866657+00:00 bl-cassoop-p11 kernel: [74739.882523] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.866658+00:00 bl-cassoop-p11 kernel: [74739.882523] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.866662+00:00 bl-cassoop-p11 kernel: [74739.882524] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:27.866663+00:00 bl-cassoop-p11 kernel: [74739.882524] Stack:
2014-01-18T13:49:27.866664+00:00 bl-cassoop-p11 kernel: [74739.882526]  ffff88301f3a3c98 ffffffff8138038a 00000000000003e9 0000000000001000
2014-01-18T13:49:27.866772+00:00 bl-cassoop-p11 kernel: [74739.882528]  ffffffff81d03da0 0000000000000400 ffff88301f3a3ca8 ffffffff813802bc
2014-01-18T13:49:27.866774+00:00 bl-cassoop-p11 kernel: [74739.882530]  ffff88301f3a3cc8 ffffffff8104383a 0000000000000002 000000000000b06a
2014-01-18T13:49:27.866775+00:00 bl-cassoop-p11 kernel: [74739.882531] Call Trace:
2014-01-18T13:49:27.866776+00:00 bl-cassoop-p11 kernel: [74739.882532]  <IRQ>
2014-01-18T13:49:27.866778+00:00 bl-cassoop-p11 kernel: [74739.882533]  [<ffffffff8138038a>] delay_tsc+0x4a/0x80
2014-01-18T13:49:27.866782+00:00 bl-cassoop-p11 kernel: [74739.882535]  [<ffffffff813802bc>] __const_udelay+0x2c/0x30
2014-01-18T13:49:27.866784+00:00 bl-cassoop-p11 kernel: [74739.882536]  [<ffffffff8104383a>] native_safe_apic_wait_icr_idle+0x2a/0x60
2014-01-18T13:49:27.866786+00:00 bl-cassoop-p11 kernel: [74739.882538]  [<ffffffff8104529f>] default_send_IPI_mask_sequence_phys+0xcf/0xe0
2014-01-18T13:49:27.866787+00:00 bl-cassoop-p11 kernel: [74739.882542]  [<ffffffff81049a57>] physflat_send_IPI_all+0x17/0x20
2014-01-18T13:49:27.866789+00:00 bl-cassoop-p11 kernel: [74739.882544]  [<ffffffff81045431>] arch_trigger_all_cpu_backtrace+0x61/0xa0
2014-01-18T13:49:27.866791+00:00 bl-cassoop-p11 kernel: [74739.882546]  [<ffffffff81106457>] print_other_cpu_stall+0x1f7/0x230
2014-01-18T13:49:27.866792+00:00 bl-cassoop-p11 kernel: [74739.882548]  [<ffffffff81106604>] __rcu_pending+0x174/0x1d0
2014-01-18T13:49:27.866796+00:00 bl-cassoop-p11 kernel: [74739.882550]  [<ffffffff81107e4d>] rcu_check_callbacks+0xed/0x1b0
2014-01-18T13:49:27.866798+00:00 bl-cassoop-p11 kernel: [74739.882553]  [<ffffffff810741b8>] update_process_times+0x48/0x80
2014-01-18T13:49:27.866800+00:00 bl-cassoop-p11 kernel: [74739.882557]  [<ffffffff810c59e3>] tick_sched_handle.isra.10+0x33/0x70
2014-01-18T13:49:27.866801+00:00 bl-cassoop-p11 kernel: [74739.882559]  [<ffffffff810c5b0c>] tick_sched_timer+0x4c/0x80
2014-01-18T13:49:27.866803+00:00 bl-cassoop-p11 kernel: [74739.882561]  [<ffffffff8108c8b8>] __run_hrtimer+0x78/0x230
2014-01-18T13:49:27.866804+00:00 bl-cassoop-p11 kernel: [74739.882563]  [<ffffffff810c5ac0>] ? tick_nohz_handler+0xa0/0xa0
2014-01-18T13:49:27.866806+00:00 bl-cassoop-p11 kernel: [74739.882564]  [<ffffffff8108d0bf>] hrtimer_interrupt+0xff/0x240
2014-01-18T13:49:27.866808+00:00 bl-cassoop-p11 kernel: [74739.882566]  [<ffffffff810437bb>] local_apic_timer_interrupt+0x3b/0x60
2014-01-18T13:49:27.866815+00:00 bl-cassoop-p11 kernel: [74739.882568]  [<ffffffff81752d25>] smp_apic_timer_interrupt+0x45/0x60
2014-01-18T13:49:27.866817+00:00 bl-cassoop-p11 kernel: [74739.882570]  [<ffffffff8175169d>] apic_timer_interrupt+0x6d/0x80
2014-01-18T13:49:27.866818+00:00 bl-cassoop-p11 kernel: [74739.882571]  <EOI>
2014-01-18T13:49:27.866820+00:00 bl-cassoop-p11 kernel: [74739.882573]  [<ffffffff810a3789>] ? enqueue_task_fair+0x2d9/0x590
2014-01-18T13:49:27.866821+00:00 bl-cassoop-p11 kernel: [74739.882576]  [<ffffffff811e5a22>] ? free_buffer_head+0x22/0x40
2014-01-18T13:49:27.866823+00:00 bl-cassoop-p11 kernel: [74739.882578]  [<ffffffff810cb4a6>] ? generic_exec_single+0x86/0xb0
2014-01-18T13:49:27.866824+00:00 bl-cassoop-p11 kernel: [74739.882579]  [<ffffffff810cb4c7>] ? generic_exec_single+0xa7/0xb0
2014-01-18T13:49:27.866828+00:00 bl-cassoop-p11 kernel: [74739.882581]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.866830+00:00 bl-cassoop-p11 kernel: [74739.882583]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.866831+00:00 bl-cassoop-p11 kernel: [74739.882585]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.866833+00:00 bl-cassoop-p11 kernel: [74739.882587]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.866834+00:00 bl-cassoop-p11 kernel: [74739.882588]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.866836+00:00 bl-cassoop-p11 kernel: [74739.882590]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.866837+00:00 bl-cassoop-p11 kernel: [74739.882592]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.866839+00:00 bl-cassoop-p11 kernel: [74739.882594]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.866843+00:00 bl-cassoop-p11 kernel: [74739.882596]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:27.866844+00:00 bl-cassoop-p11 kernel: [74739.882598]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:49:27.866846+00:00 bl-cassoop-p11 kernel: [74739.882600]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:27.866848+00:00 bl-cassoop-p11 kernel: [74739.882602]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:27.866849+00:00 bl-cassoop-p11 kernel: [74739.882604]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:27.866851+00:00 bl-cassoop-p11 kernel: [74739.882606]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:27.866852+00:00 bl-cassoop-p11 kernel: [74739.882608]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.866855+00:00 bl-cassoop-p11 kernel: [74739.882610]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.866857+00:00 bl-cassoop-p11 kernel: [74739.882612]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.866859+00:00 bl-cassoop-p11 kernel: [74739.882613]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:27.866860+00:00 bl-cassoop-p11 kernel: [74739.882615]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:27.866862+00:00 bl-cassoop-p11 kernel: [74739.882617]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:27.866863+00:00 bl-cassoop-p11 kernel: [74739.882619]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:27.866865+00:00 bl-cassoop-p11 kernel: [74739.882621]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:27.866867+00:00 bl-cassoop-p11 kernel: [74739.882623]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:49:27.866870+00:00 bl-cassoop-p11 kernel: [74739.882625]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:27.866872+00:00 bl-cassoop-p11 kernel: [74739.882627]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:27.866873+00:00 bl-cassoop-p11 kernel: [74739.882629]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.866875+00:00 bl-cassoop-p11 kernel: [74739.882631]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.866877+00:00 bl-cassoop-p11 kernel: [74739.882633]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:27.866878+00:00 bl-cassoop-p11 kernel: [74739.882635]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:27.866880+00:00 bl-cassoop-p11 kernel: [74739.882636]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:27.866883+00:00 bl-cassoop-p11 kernel: [74739.882638]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.866885+00:00 bl-cassoop-p11 kernel: [74739.882640]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:27.866983+00:00 bl-cassoop-p11 kernel: [74739.882642]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.866984+00:00 bl-cassoop-p11 kernel: [74739.882643]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.866986+00:00 bl-cassoop-p11 kernel: [74739.882645]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.866987+00:00 bl-cassoop-p11 kernel: [74739.882647]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:27.866989+00:00 bl-cassoop-p11 kernel: [74739.882648]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:27.866991+00:00 bl-cassoop-p11 kernel: [74739.882650]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.866994+00:00 bl-cassoop-p11 kernel: [74739.882652]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:27.866995+00:00 bl-cassoop-p11 kernel: [74739.882653]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.866997+00:00 bl-cassoop-p11 kernel: [74739.882655]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.867000+00:00 bl-cassoop-p11 kernel: [74739.882666] Code: e8 4c 8b 65 f0 4c 8b 6d f8 c9 c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 5d c3 90 90 90 90 90 55 48 89 e5 0f 31 <89> c1 48 89 d0 48 c1 e0 20 48 09 c8 5d c3 66 66 66 2e 0f 1f 84
2014-01-18T13:49:27.867001+00:00 bl-cassoop-p11 kernel: [74739.882672] NMI backtrace for cpu 28
2014-01-18T13:49:27.867003+00:00 bl-cassoop-p11 kernel: [74739.882674] CPU: 28 PID: 0 Comm: swapper/28 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867004+00:00 bl-cassoop-p11 kernel: [74739.882675] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867008+00:00 bl-cassoop-p11 kernel: [74739.882676] task: ffff8817fadcddc0 ti: ffff8817fadd8000 task.ti: ffff8817fadd8000
2014-01-18T13:49:27.867010+00:00 bl-cassoop-p11 kernel: [74739.882679] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.867011+00:00 bl-cassoop-p11 kernel: [74739.882680] RSP: 0018:ffff8817fadd9da8  EFLAGS: 00000046
2014-01-18T13:49:27.867012+00:00 bl-cassoop-p11 kernel: [74739.882681] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.867014+00:00 bl-cassoop-p11 kernel: [74739.882682] RDX: 0000000000000000 RSI: ffff8817fadd9fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.867015+00:00 bl-cassoop-p11 kernel: [74739.882683] RBP: ffff8817fadd9dd8 R08: 0000000000001445 R09: 000000000000a256
2014-01-18T13:49:27.867017+00:00 bl-cassoop-p11 kernel: [74739.882683] R10: 0000000000001c1a R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.867020+00:00 bl-cassoop-p11 kernel: [74739.882684] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.867022+00:00 bl-cassoop-p11 kernel: [74739.882685] FS:  0000000000000000(0000) GS:ffff88181fbc0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867023+00:00 bl-cassoop-p11 kernel: [74739.882686] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867025+00:00 bl-cassoop-p11 kernel: [74739.882687] CR2: 000000008b06d0f0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867026+00:00 bl-cassoop-p11 kernel: [74739.882688] Stack:
2014-01-18T13:49:27.867027+00:00 bl-cassoop-p11 kernel: [74739.882692]  ffff8817fadd9dd8 0000001c810bc824 ffffe8e8003c0320 000043f363d8ed4a
2014-01-18T13:49:27.867029+00:00 bl-cassoop-p11 kernel: [74739.882695]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd9e38 ffffffff815e5a9f
2014-01-18T13:49:27.867030+00:00 bl-cassoop-p11 kernel: [74739.882698]  0000000000000000 00000000019cfb8b 0000000000000000 00000000019cfb8b
2014-01-18T13:49:27.867033+00:00 bl-cassoop-p11 kernel: [74739.882699] Call Trace:
2014-01-18T13:49:27.867034+00:00 bl-cassoop-p11 kernel: [74739.882701]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.867036+00:00 bl-cassoop-p11 kernel: [74739.882703]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.867037+00:00 bl-cassoop-p11 kernel: [74739.882706]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.867039+00:00 bl-cassoop-p11 kernel: [74739.882709]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.867041+00:00 bl-cassoop-p11 kernel: [74739.882711]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.867042+00:00 bl-cassoop-p11 kernel: [74739.882714]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.867046+00:00 bl-cassoop-p11 kernel: [74739.882733] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.867048+00:00 bl-cassoop-p11 kernel: [74739.882772] NMI backtrace for cpu 29
2014-01-18T13:49:27.867050+00:00 bl-cassoop-p11 kernel: [74739.882774] CPU: 29 PID: 0 Comm: swapper/29 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867051+00:00 bl-cassoop-p11 kernel: [74739.882775] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867053+00:00 bl-cassoop-p11 kernel: [74739.882776] task: ffff8817fade0000 ti: ffff8817fadda000 task.ti: ffff8817fadda000
2014-01-18T13:49:27.867055+00:00 bl-cassoop-p11 kernel: [74739.882779] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.867056+00:00 bl-cassoop-p11 kernel: [74739.882780] RSP: 0018:ffff8817faddbda8  EFLAGS: 00000046
2014-01-18T13:49:27.867066+00:00 bl-cassoop-p11 kernel: [74739.882780] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.867068+00:00 bl-cassoop-p11 kernel: [74739.882781] RDX: 0000000000000000 RSI: ffff8817faddbfd8 RDI: 000000000000001d
2014-01-18T13:49:27.867069+00:00 bl-cassoop-p11 kernel: [74739.882782] RBP: ffff8817faddbdd8 R08: 000000000001b7f5 R09: 000000001b01a9a4
2014-01-18T13:49:27.867071+00:00 bl-cassoop-p11 kernel: [74739.882783] R10: 0000000000001ff9 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.867072+00:00 bl-cassoop-p11 kernel: [74739.882784] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.867073+00:00 bl-cassoop-p11 kernel: [74739.882785] FS:  0000000000000000(0000) GS:ffff88301f3c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867075+00:00 bl-cassoop-p11 kernel: [74739.882786] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867076+00:00 bl-cassoop-p11 kernel: [74739.882787] CR2: 00007f0b82eaf1a8 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867080+00:00 bl-cassoop-p11 kernel: [74739.882787] Stack:
2014-01-18T13:49:27.867187+00:00 bl-cassoop-p11 kernel: [74739.882791]  ffff8817faddbdd8 0000001d810bc824 ffffe8ffffbc0320 000043f36055bd7a
2014-01-18T13:49:27.867188+00:00 bl-cassoop-p11 kernel: [74739.882794]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817faddbe38 ffffffff815e5a9f
2014-01-18T13:49:27.867190+00:00 bl-cassoop-p11 kernel: [74739.882797]  0000000000000000 00000000054bcb82 0000000000000000 00000000054bcb82
2014-01-18T13:49:27.867191+00:00 bl-cassoop-p11 kernel: [74739.882798] Call Trace:
2014-01-18T13:49:27.867192+00:00 bl-cassoop-p11 kernel: [74739.882800]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.867194+00:00 bl-cassoop-p11 kernel: [74739.882802]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.867198+00:00 bl-cassoop-p11 kernel: [74739.882805]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.867199+00:00 bl-cassoop-p11 kernel: [74739.882807]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.867201+00:00 bl-cassoop-p11 kernel: [74739.882810]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.867203+00:00 bl-cassoop-p11 kernel: [74739.882812]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.867205+00:00 bl-cassoop-p11 kernel: [74739.882831] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.867206+00:00 bl-cassoop-p11 kernel: [74739.882870] NMI backtrace for cpu 30
2014-01-18T13:49:27.867208+00:00 bl-cassoop-p11 kernel: [74739.882872] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867211+00:00 bl-cassoop-p11 kernel: [74739.882873] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867213+00:00 bl-cassoop-p11 kernel: [74739.882874] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:49:27.867215+00:00 bl-cassoop-p11 kernel: [74739.882876] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:27.867216+00:00 bl-cassoop-p11 kernel: [74739.882877] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:49:27.867218+00:00 bl-cassoop-p11 kernel: [74739.882878] RAX: 0000000000000286 RBX: ffff88188471f8d0 RCX: 0000000000000001
2014-01-18T13:49:27.867219+00:00 bl-cassoop-p11 kernel: [74739.882879] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:27.867221+00:00 bl-cassoop-p11 kernel: [74739.882879] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:49:27.867222+00:00 bl-cassoop-p11 kernel: [74739.882880] R10: 00000000000035de R11: 0000000000000001 R12: 000000000000000f
2014-01-18T13:49:27.867225+00:00 bl-cassoop-p11 kernel: [74739.882881] R13: ffff88301f2f5040 R14: ffff88301f2f5050 R15: 0000000000000001
2014-01-18T13:49:27.867227+00:00 bl-cassoop-p11 kernel: [74739.882882] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867228+00:00 bl-cassoop-p11 kernel: [74739.882883] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867230+00:00 bl-cassoop-p11 kernel: [74739.882884] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867231+00:00 bl-cassoop-p11 kernel: [74739.882885] Stack:
2014-01-18T13:49:27.867232+00:00 bl-cassoop-p11 kernel: [74739.882889]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:49:27.867234+00:00 bl-cassoop-p11 kernel: [74739.882892]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:49:27.867237+00:00 bl-cassoop-p11 kernel: [74739.882895]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:49:27.867238+00:00 bl-cassoop-p11 kernel: [74739.882896] Call Trace:
2014-01-18T13:49:27.867240+00:00 bl-cassoop-p11 kernel: [74739.882898]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.867241+00:00 bl-cassoop-p11 kernel: [74739.882901]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.867243+00:00 bl-cassoop-p11 kernel: [74739.882903]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.867244+00:00 bl-cassoop-p11 kernel: [74739.882906]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.867246+00:00 bl-cassoop-p11 kernel: [74739.882908]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.867248+00:00 bl-cassoop-p11 kernel: [74739.882910]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:49:27.867251+00:00 bl-cassoop-p11 kernel: [74739.882913]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.867253+00:00 bl-cassoop-p11 kernel: [74739.882916]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.867254+00:00 bl-cassoop-p11 kernel: [74739.882918]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:49:27.867256+00:00 bl-cassoop-p11 kernel: [74739.882920]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:49:27.867258+00:00 bl-cassoop-p11 kernel: [74739.882923]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:49:27.867259+00:00 bl-cassoop-p11 kernel: [74739.882925]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:49:27.867261+00:00 bl-cassoop-p11 kernel: [74739.882927]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:49:27.867264+00:00 bl-cassoop-p11 kernel: [74739.882929]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:49:27.867266+00:00 bl-cassoop-p11 kernel: [74739.882931]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:49:27.867267+00:00 bl-cassoop-p11 kernel: [74739.882934]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:49:27.867269+00:00 bl-cassoop-p11 kernel: [74739.882936]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:49:27.867270+00:00 bl-cassoop-p11 kernel: [74739.882938]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:49:27.867272+00:00 bl-cassoop-p11 kernel: [74739.882941]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:49:27.867273+00:00 bl-cassoop-p11 kernel: [74739.882943]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:49:27.867275+00:00 bl-cassoop-p11 kernel: [74739.882946]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:49:27.867278+00:00 bl-cassoop-p11 kernel: [74739.882948]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:27.867280+00:00 bl-cassoop-p11 kernel: [74739.882951]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:27.867281+00:00 bl-cassoop-p11 kernel: [74739.882953]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:27.867283+00:00 bl-cassoop-p11 kernel: [74739.882956]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:27.867370+00:00 bl-cassoop-p11 kernel: [74739.882958]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:49:27.867372+00:00 bl-cassoop-p11 kernel: [74739.882961]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:27.867373+00:00 bl-cassoop-p11 kernel: [74739.882964]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:49:27.867377+00:00 bl-cassoop-p11 kernel: [74739.882982] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:27.867379+00:00 bl-cassoop-p11 kernel: [74739.882984] NMI backtrace for cpu 31
2014-01-18T13:49:27.867380+00:00 bl-cassoop-p11 kernel: [74739.882986] CPU: 31 PID: 0 Comm: swapper/31 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867382+00:00 bl-cassoop-p11 kernel: [74739.882987] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867384+00:00 bl-cassoop-p11 kernel: [74739.882988] task: ffff8817fade2ee0 ti: ffff8817fadde000 task.ti: ffff8817fadde000
2014-01-18T13:49:27.867386+00:00 bl-cassoop-p11 kernel: [74739.882991] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.867387+00:00 bl-cassoop-p11 kernel: [74739.882992] RSP: 0018:ffff8817faddfda8  EFLAGS: 00000046
2014-01-18T13:49:27.867390+00:00 bl-cassoop-p11 kernel: [74739.882993] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.867392+00:00 bl-cassoop-p11 kernel: [74739.882994] RDX: 0000000000000000 RSI: ffff8817faddffd8 RDI: 000000000000001f
2014-01-18T13:49:27.867393+00:00 bl-cassoop-p11 kernel: [74739.882994] RBP: ffff8817faddfdd8 R08: 0000000000016cdd R09: 0000000069d6f5d9
2014-01-18T13:49:27.867395+00:00 bl-cassoop-p11 kernel: [74739.882995] R10: 00000000000018f1 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.867396+00:00 bl-cassoop-p11 kernel: [74739.882996] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.867397+00:00 bl-cassoop-p11 kernel: [74739.882997] FS:  0000000000000000(0000) GS:ffff88301f3e0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867399+00:00 bl-cassoop-p11 kernel: [74739.882998] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867400+00:00 bl-cassoop-p11 kernel: [74739.882999] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867403+00:00 bl-cassoop-p11 kernel: [74739.882999] Stack:
2014-01-18T13:49:27.867405+00:00 bl-cassoop-p11 kernel: [74739.883003]  ffff8817faddfdd8 0000001f810bc824 ffffe8ffffbe0320 000043f361bf36cc
2014-01-18T13:49:27.867406+00:00 bl-cassoop-p11 kernel: [74739.883007]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817faddfe38 ffffffff815e5a9f
2014-01-18T13:49:27.867408+00:00 bl-cassoop-p11 kernel: [74739.883010]  0000000000000000 00000000083a60b7 0000000000000000 00000000083a60b7
2014-01-18T13:49:27.867409+00:00 bl-cassoop-p11 kernel: [74739.883010] Call Trace:
2014-01-18T13:49:27.867410+00:00 bl-cassoop-p11 kernel: [74739.883013]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.867412+00:00 bl-cassoop-p11 kernel: [74739.883015]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.867415+00:00 bl-cassoop-p11 kernel: [74739.883017]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.867417+00:00 bl-cassoop-p11 kernel: [74739.883020]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.867418+00:00 bl-cassoop-p11 kernel: [74739.883022]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.867420+00:00 bl-cassoop-p11 kernel: [74739.883025]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.867422+00:00 bl-cassoop-p11 kernel: [74739.883043] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.867424+00:00 bl-cassoop-p11 kernel: [74739.889344] sending NMI to all CPUs:
2014-01-18T13:49:27.867425+00:00 bl-cassoop-p11 kernel: [74739.889347] NMI backtrace for cpu 0
2014-01-18T13:49:27.867428+00:00 bl-cassoop-p11 kernel: [74739.889350] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867430+00:00 bl-cassoop-p11 kernel: [74739.889352] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867431+00:00 bl-cassoop-p11 kernel: [74739.889354] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:49:27.867433+00:00 bl-cassoop-p11 kernel: [74739.889355] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.867435+00:00 bl-cassoop-p11 kernel: [74739.889358] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:49:27.867436+00:00 bl-cassoop-p11 kernel: [74739.889360] RAX: 0000000000000008 RBX: ffff88181fa15088 RCX: ffff88181fa97868
2014-01-18T13:49:27.867437+00:00 bl-cassoop-p11 kernel: [74739.889362] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.867441+00:00 bl-cassoop-p11 kernel: [74739.889363] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:49:27.867442+00:00 bl-cassoop-p11 kernel: [74739.889365] R10: 0000000000000004 R11: 0000000000000206 R12: ffff88181fa15080
2014-01-18T13:49:27.867444+00:00 bl-cassoop-p11 kernel: [74739.889367] R13: ffffffff8105ae90 R14: ffff880045ccfd50 R15: 0000000000015040
2014-01-18T13:49:27.867445+00:00 bl-cassoop-p11 kernel: [74739.889369] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867447+00:00 bl-cassoop-p11 kernel: [74739.889370] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867448+00:00 bl-cassoop-p11 kernel: [74739.889372] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:49:27.867449+00:00 bl-cassoop-p11 kernel: [74739.889373] Stack:
2014-01-18T13:49:27.867450+00:00 bl-cassoop-p11 kernel: [74739.889374]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:49:27.867455+00:00 bl-cassoop-p11 kernel: [74739.889380]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:49:27.867457+00:00 bl-cassoop-p11 kernel: [74739.889384]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:49:27.867458+00:00 bl-cassoop-p11 kernel: [74739.889389] Call Trace:
2014-01-18T13:49:27.867518+00:00 bl-cassoop-p11 kernel: [74739.889393]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.867520+00:00 bl-cassoop-p11 kernel: [74739.889396]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.867522+00:00 bl-cassoop-p11 kernel: [74739.889400]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:27.867523+00:00 bl-cassoop-p11 kernel: [74739.889403]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:27.867526+00:00 bl-cassoop-p11 kernel: [74739.889406]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:27.867528+00:00 bl-cassoop-p11 kernel: [74739.889410]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:27.867530+00:00 bl-cassoop-p11 kernel: [74739.889413]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:27.867531+00:00 bl-cassoop-p11 kernel: [74739.889416]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:27.867533+00:00 bl-cassoop-p11 kernel: [74739.889420]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.867535+00:00 bl-cassoop-p11 kernel: [74739.889421] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.867536+00:00 bl-cassoop-p11 kernel: [74739.889453] NMI backtrace for cpu 1
2014-01-18T13:49:27.867538+00:00 bl-cassoop-p11 kernel: [74739.889456] CPU: 1 PID: 5830 Comm: collectd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867542+00:00 bl-cassoop-p11 kernel: [74739.889458] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867543+00:00 bl-cassoop-p11 kernel: [74739.889460] task: ffff882ff0912ee0 ti: ffff882ffa6f6000 task.ti: ffff882ffa6f6000
2014-01-18T13:49:27.867545+00:00 bl-cassoop-p11 kernel: [74739.889461] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:27.867547+00:00 bl-cassoop-p11 kernel: [74739.889465] RSP: 0018:ffff882ffa6f7d08  EFLAGS: 00000202
2014-01-18T13:49:27.867548+00:00 bl-cassoop-p11 kernel: [74739.889466] RAX: 0000000000000286 RBX: ffff882ffa6f7d70 RCX: 0000000000000001
2014-01-18T13:49:27.867550+00:00 bl-cassoop-p11 kernel: [74739.889468] RDX: ffff88181fa38680 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:27.867551+00:00 bl-cassoop-p11 kernel: [74739.889469] RBP: ffff882ffa6f7d48 R08: 0000000000000000 R09: 000000000000730e
2014-01-18T13:49:27.867554+00:00 bl-cassoop-p11 kernel: [74739.889471] R10: 0000000000003708 R11: 0000000000000293 R12: 0000000000000002
2014-01-18T13:49:27.867556+00:00 bl-cassoop-p11 kernel: [74739.889472] R13: ffff88181fa35040 R14: ffff88181fa35050 R15: 0000000000000001
2014-01-18T13:49:27.867557+00:00 bl-cassoop-p11 kernel: [74739.889474] FS:  00007f2a8a89f700(0000) GS:ffff88301f200000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867559+00:00 bl-cassoop-p11 kernel: [74739.889476] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867560+00:00 bl-cassoop-p11 kernel: [74739.889477] CR2: 00007f2a92182000 CR3: 0000002ff0b0e000 CR4: 00000000000407e0
2014-01-18T13:49:27.867561+00:00 bl-cassoop-p11 kernel: [74739.889479] Stack:
2014-01-18T13:49:27.867563+00:00 bl-cassoop-p11 kernel: [74739.889480]  ffff882ff7817000 ffff88181fa386a8 00000007f2a9217e 0000000000000002
2014-01-18T13:49:27.867566+00:00 bl-cassoop-p11 kernel: [74739.889485]  ffffffff8139a810 0000000000000001 ffffffff81d03da0 00007f2a92182000
2014-01-18T13:49:27.867568+00:00 bl-cassoop-p11 kernel: [74739.889490]  ffff882ffa6f7db8 ffffffff810cb5a5 ffff882ff0912ee0 0000000000000001
2014-01-18T13:49:27.867568+00:00 bl-cassoop-p11 kernel: [74739.889494] Call Trace:
2014-01-18T13:49:27.867570+00:00 bl-cassoop-p11 kernel: [74739.889497]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:27.867572+00:00 bl-cassoop-p11 kernel: [74739.889500]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.867573+00:00 bl-cassoop-p11 kernel: [74739.889503]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:27.867575+00:00 bl-cassoop-p11 kernel: [74739.889506]  [<ffffffff8139ab4e>] rdmsr_on_cpu+0x4e/0x70
2014-01-18T13:49:27.867577+00:00 bl-cassoop-p11 kernel: [74739.889510]  [<ffffffffa001b62b>] show_temp+0xab/0xf0 [coretemp]
2014-01-18T13:49:27.867584+00:00 bl-cassoop-p11 kernel: [74739.889514]  [<ffffffff81490f20>] dev_attr_show+0x20/0x60
2014-01-18T13:49:27.867586+00:00 bl-cassoop-p11 kernel: [74739.889517]  [<ffffffff8114d37e>] ? __get_free_pages+0xe/0x40
2014-01-18T13:49:27.867587+00:00 bl-cassoop-p11 kernel: [74739.889520]  [<ffffffff81227e86>] fill_read_buffer.isra.9+0x66/0xf0
2014-01-18T13:49:27.867589+00:00 bl-cassoop-p11 kernel: [74739.889523]  [<ffffffff81227fc0>] sysfs_read_file+0xb0/0xf0
2014-01-18T13:49:27.867590+00:00 bl-cassoop-p11 kernel: [74739.889527]  [<ffffffff811b42e4>] vfs_read+0xb4/0x180
2014-01-18T13:49:27.867592+00:00 bl-cassoop-p11 kernel: [74739.889530]  [<ffffffff811b4562>] SyS_read+0x52/0xa0
2014-01-18T13:49:27.867593+00:00 bl-cassoop-p11 kernel: [74739.889534]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.867598+00:00 bl-cassoop-p11 kernel: [74739.889535] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:27.867599+00:00 bl-cassoop-p11 kernel: [74739.889566] NMI backtrace for cpu 2
2014-01-18T13:49:27.867601+00:00 bl-cassoop-p11 kernel: [74739.889569] CPU: 2 PID: 42018 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867602+00:00 bl-cassoop-p11 kernel: [74739.889570] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867604+00:00 bl-cassoop-p11 kernel: [74739.889572] task: ffff881881ad4650 ti: ffff88010dc50000 task.ti: ffff88010dc50000
2014-01-18T13:49:27.867606+00:00 bl-cassoop-p11 kernel: [74739.889574] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.867607+00:00 bl-cassoop-p11 kernel: [74739.889577] RSP: 0018:ffff88010dc51e68  EFLAGS: 00000087
2014-01-18T13:49:27.867611+00:00 bl-cassoop-p11 kernel: [74739.889579] RAX: ffff88010dc51ea0 RBX: ffff881881ad4650 RCX: 0000000000000001
2014-01-18T13:49:27.867612+00:00 bl-cassoop-p11 kernel: [74739.889580] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.867614+00:00 bl-cassoop-p11 kernel: [74739.889582] RBP: ffff88010dc51e68 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.867716+00:00 bl-cassoop-p11 kernel: [74739.889584] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:27.867718+00:00 bl-cassoop-p11 kernel: [74739.889585] R13: ffff881881ad4650 R14: 0000000000000000 R15: 0000000000000003
2014-01-18T13:49:27.867719+00:00 bl-cassoop-p11 kernel: [74739.889587] FS:  0000000000000000(0000) GS:ffff88181fa20000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867721+00:00 bl-cassoop-p11 kernel: [74739.889589] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867722+00:00 bl-cassoop-p11 kernel: [74739.889591] CR2: 00007eff09a77530 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.867725+00:00 bl-cassoop-p11 kernel: [74739.889592] Stack:
2014-01-18T13:49:27.867727+00:00 bl-cassoop-p11 kernel: [74739.889593]  ffff88010dc51e78 ffffffff8174787e ffff88010dc51ee8 ffffffff810676e5
2014-01-18T13:49:27.867728+00:00 bl-cassoop-p11 kernel: [74739.889598]  ffff88010dc51f08 ffff88010dc51ea0 ffff881881ad4650 ffff88010dc51ea0
2014-01-18T13:49:27.867730+00:00 bl-cassoop-p11 kernel: [74739.889603]  ffff88010dc51ea0 0000000100000000 0001c78d00000001 ffff881881ad4650
2014-01-18T13:49:27.867731+00:00 bl-cassoop-p11 kernel: [74739.889607] Call Trace:
2014-01-18T13:49:27.867732+00:00 bl-cassoop-p11 kernel: [74739.889611]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.867734+00:00 bl-cassoop-p11 kernel: [74739.889614]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.867737+00:00 bl-cassoop-p11 kernel: [74739.889617]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.867739+00:00 bl-cassoop-p11 kernel: [74739.889620]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.867740+00:00 bl-cassoop-p11 kernel: [74739.889623]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:27.867742+00:00 bl-cassoop-p11 kernel: [74739.889627]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.867745+00:00 bl-cassoop-p11 kernel: [74739.889628] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.867746+00:00 bl-cassoop-p11 kernel: [74739.889659] NMI backtrace for cpu 3
2014-01-18T13:49:27.867748+00:00 bl-cassoop-p11 kernel: [74739.889662] CPU: 3 PID: 5299 Comm: nrpe Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867751+00:00 bl-cassoop-p11 kernel: [74739.889664] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867752+00:00 bl-cassoop-p11 kernel: [74739.889665] task: ffff882ff17d8000 ti: ffff882ff0376000 task.ti: ffff882ff0376000
2014-01-18T13:49:27.867754+00:00 bl-cassoop-p11 kernel: [74739.889666] RIP: 0010:[<ffffffff813817cd>]  [<ffffffff813817cd>] __write_lock_failed+0xd/0x20
2014-01-18T13:49:27.867756+00:00 bl-cassoop-p11 kernel: [74739.889669] RSP: 0018:ffff882ff0377e10  EFLAGS: 00000087
2014-01-18T13:49:27.867757+00:00 bl-cassoop-p11 kernel: [74739.889670] RAX: 000000000000a638 RBX: ffff881880f29770 RCX: 0000000000000000
2014-01-18T13:49:27.867759+00:00 bl-cassoop-p11 kernel: [74739.889671] RDX: 0000000000000011 RSI: 0000000000000006 RDI: ffffffff81c06040
2014-01-18T13:49:27.867760+00:00 bl-cassoop-p11 kernel: [74739.889672] RBP: ffff882ff0377e10 R08: ffff8823fe8717b0 R09: 0000000000000040
2014-01-18T13:49:27.867762+00:00 bl-cassoop-p11 kernel: [74739.889673] R10: ffff8823fe8717c0 R11: 0000000000000000 R12: 0000000001200011
2014-01-18T13:49:27.867765+00:00 bl-cassoop-p11 kernel: [74739.889674] R13: ffff881880f29b00 R14: 0000000000000000 R15: 00007f451b60b9d0
2014-01-18T13:49:27.867766+00:00 bl-cassoop-p11 kernel: [74739.889675] FS:  00007f451b60b700(0000) GS:ffff88301f220000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867768+00:00 bl-cassoop-p11 kernel: [74739.889676] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867769+00:00 bl-cassoop-p11 kernel: [74739.889677] CR2: 000000079d884bfc CR3: 0000002ff7997000 CR4: 00000000000407e0
2014-01-18T13:49:27.867770+00:00 bl-cassoop-p11 kernel: [74739.889678] Stack:
2014-01-18T13:49:27.867772+00:00 bl-cassoop-p11 kernel: [74739.889679]  ffff882ff0377e20 ffffffff8174787e ffff882ff0377ea0 ffffffff81063fda
2014-01-18T13:49:27.867773+00:00 bl-cassoop-p11 kernel: [74739.889683]  fffffffffffffff5 0000000000000000 ffff881880f29a80 0000000000000005
2014-01-18T13:49:27.867776+00:00 bl-cassoop-p11 kernel: [74739.889685]  0000000000000000 ffff8823fe871780 ffffffff811d1165 00000000f0377f58
2014-01-18T13:49:27.867778+00:00 bl-cassoop-p11 kernel: [74739.889688] Call Trace:
2014-01-18T13:49:27.867779+00:00 bl-cassoop-p11 kernel: [74739.889691]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.867781+00:00 bl-cassoop-p11 kernel: [74739.889693]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:27.867782+00:00 bl-cassoop-p11 kernel: [74739.889695]  [<ffffffff811d1165>] ? fd_install+0x25/0x30
2014-01-18T13:49:27.867784+00:00 bl-cassoop-p11 kernel: [74739.889697]  [<ffffffff8162162a>] ? SYSC_accept4+0x17a/0x240
2014-01-18T13:49:27.867786+00:00 bl-cassoop-p11 kernel: [74739.889699]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:27.867787+00:00 bl-cassoop-p11 kernel: [74739.889701]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:27.867790+00:00 bl-cassoop-p11 kernel: [74739.889703]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:27.867792+00:00 bl-cassoop-p11 kernel: [74739.889705]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:27.867793+00:00 bl-cassoop-p11 kernel: [74739.889707]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.867796+00:00 bl-cassoop-p11 kernel: [74739.889708] Code: 01 31 c0 66 66 90 c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 <81> 3f 00 00 10 00 75 f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48
2014-01-18T13:49:27.867797+00:00 bl-cassoop-p11 kernel: [74739.889745] NMI backtrace for cpu 4
2014-01-18T13:49:27.867799+00:00 bl-cassoop-p11 kernel: [74739.889749] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867800+00:00 bl-cassoop-p11 kernel: [74739.889751] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867805+00:00 bl-cassoop-p11 kernel: [74739.889753] task: ffff8817fad80000 ti: ffff8817fad88000 task.ti: ffff8817fad88000
2014-01-18T13:49:27.867807+00:00 bl-cassoop-p11 kernel: [74739.889754] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.867873+00:00 bl-cassoop-p11 kernel: [74739.889758] RSP: 0018:ffff8817fad89da8  EFLAGS: 00000046
2014-01-18T13:49:27.867875+00:00 bl-cassoop-p11 kernel: [74739.889759] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.867876+00:00 bl-cassoop-p11 kernel: [74739.889761] RDX: 0000000000000000 RSI: ffff8817fad89fd8 RDI: 0000000000000004
2014-01-18T13:49:27.867877+00:00 bl-cassoop-p11 kernel: [74739.889762] RBP: ffff8817fad89dd8 R08: 00000000000082e9 R09: 000000000e268900
2014-01-18T13:49:27.867879+00:00 bl-cassoop-p11 kernel: [74739.889764] R10: 0000000000001de0 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.867882+00:00 bl-cassoop-p11 kernel: [74739.889766] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.867884+00:00 bl-cassoop-p11 kernel: [74739.889768] FS:  0000000000000000(0000) GS:ffff88181fa40000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867885+00:00 bl-cassoop-p11 kernel: [74739.889769] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867887+00:00 bl-cassoop-p11 kernel: [74739.889771] CR2: 000000008b06c0d0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867888+00:00 bl-cassoop-p11 kernel: [74739.889772] Stack:
2014-01-18T13:49:27.867889+00:00 bl-cassoop-p11 kernel: [74739.889773]  ffff8817fad89dd8 00000004810bc824 ffffe8e800240320 000043f364dfa873
2014-01-18T13:49:27.867891+00:00 bl-cassoop-p11 kernel: [74739.889778]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad89e38 ffffffff815e5a9f
2014-01-18T13:49:27.867892+00:00 bl-cassoop-p11 kernel: [74739.889783]  0000000000000000 0000000006f5438d 0000000000000000 0000000006f5438d
2014-01-18T13:49:27.867895+00:00 bl-cassoop-p11 kernel: [74739.889787] Call Trace:
2014-01-18T13:49:27.867896+00:00 bl-cassoop-p11 kernel: [74739.889791]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.867898+00:00 bl-cassoop-p11 kernel: [74739.889794]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.867900+00:00 bl-cassoop-p11 kernel: [74739.889797]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.867901+00:00 bl-cassoop-p11 kernel: [74739.889801]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.867903+00:00 bl-cassoop-p11 kernel: [74739.889804]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.867904+00:00 bl-cassoop-p11 kernel: [74739.889808]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.867909+00:00 bl-cassoop-p11 kernel: [74739.889809] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.867910+00:00 bl-cassoop-p11 kernel: [74739.889871] NMI backtrace for cpu 5
2014-01-18T13:49:27.867912+00:00 bl-cassoop-p11 kernel: [74739.889883] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867913+00:00 bl-cassoop-p11 kernel: [74739.889892] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867915+00:00 bl-cassoop-p11 kernel: [74739.889902] task: ffff8817fad81770 ti: ffff8817fad8a000 task.ti: ffff8817fad8a000
2014-01-18T13:49:27.867917+00:00 bl-cassoop-p11 kernel: [74739.889911] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.867918+00:00 bl-cassoop-p11 kernel: [74739.889928] RSP: 0018:ffff8817fad8bda8  EFLAGS: 00000046
2014-01-18T13:49:27.867921+00:00 bl-cassoop-p11 kernel: [74739.889938] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.867923+00:00 bl-cassoop-p11 kernel: [74739.889947] RDX: 0000000000000000 RSI: ffff8817fad8bfd8 RDI: 0000000000000005
2014-01-18T13:49:27.867924+00:00 bl-cassoop-p11 kernel: [74739.889952] RBP: ffff8817fad8bdd8 R08: 0000000000005409 R09: 000000000109b729
2014-01-18T13:49:27.867926+00:00 bl-cassoop-p11 kernel: [74739.889953] R10: 0000000000001d4e R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.867927+00:00 bl-cassoop-p11 kernel: [74739.889955] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.867929+00:00 bl-cassoop-p11 kernel: [74739.889956] FS:  0000000000000000(0000) GS:ffff88301f240000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.867930+00:00 bl-cassoop-p11 kernel: [74739.889957] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.867932+00:00 bl-cassoop-p11 kernel: [74739.889958] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.867934+00:00 bl-cassoop-p11 kernel: [74739.889959] Stack:
2014-01-18T13:49:27.867936+00:00 bl-cassoop-p11 kernel: [74739.889960]  ffff8817fad8bdd8 00000005810bc824 ffffe8ffffa40320 000043f364e12545
2014-01-18T13:49:27.867937+00:00 bl-cassoop-p11 kernel: [74739.889964]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad8be38 ffffffff815e5a9f
2014-01-18T13:49:27.867939+00:00 bl-cassoop-p11 kernel: [74739.889967]  0000000000000000 000000000cb6bd45 0000000000000000 000000000cb6bd45
2014-01-18T13:49:27.867940+00:00 bl-cassoop-p11 kernel: [74739.889970] Call Trace:
2014-01-18T13:49:27.867942+00:00 bl-cassoop-p11 kernel: [74739.889973]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.867943+00:00 bl-cassoop-p11 kernel: [74739.889975]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.867947+00:00 bl-cassoop-p11 kernel: [74739.889978]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.867949+00:00 bl-cassoop-p11 kernel: [74739.889980]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.867950+00:00 bl-cassoop-p11 kernel: [74739.889983]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.867952+00:00 bl-cassoop-p11 kernel: [74739.889986]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.867954+00:00 bl-cassoop-p11 kernel: [74739.889987] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.867956+00:00 bl-cassoop-p11 kernel: [74739.890008] NMI backtrace for cpu 6
2014-01-18T13:49:27.867957+00:00 bl-cassoop-p11 kernel: [74739.890011] CPU: 6 PID: 5135 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.867965+00:00 bl-cassoop-p11 kernel: [74739.890012] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.867967+00:00 bl-cassoop-p11 kernel: [74739.890013] task: ffff8817f9b4ddc0 ti: ffff8817f927c000 task.ti: ffff8817f927c000
2014-01-18T13:49:27.868072+00:00 bl-cassoop-p11 kernel: [74739.890014] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.868074+00:00 bl-cassoop-p11 kernel: [74739.890017] RSP: 0018:ffff8817f927de10  EFLAGS: 00000087
2014-01-18T13:49:27.868075+00:00 bl-cassoop-p11 kernel: [74739.890019] RAX: 0000000000001382 RBX: ffff88171d6bddc0 RCX: 0000000000000000
2014-01-18T13:49:27.868077+00:00 bl-cassoop-p11 kernel: [74739.890020] RDX: ffff8817f3792ee0 RSI: 0000000000000006 RDI: ffffffff81c06040
2014-01-18T13:49:27.868078+00:00 bl-cassoop-p11 kernel: [74739.890021] RBP: ffff8817f927de10 R08: ffff8817f7bd2430 R09: 0000000000000040
2014-01-18T13:49:27.868079+00:00 bl-cassoop-p11 kernel: [74739.890022] R10: ffff8817f7bd2440 R11: e000000000000000 R12: 00000000003d0f00
2014-01-18T13:49:27.868084+00:00 bl-cassoop-p11 kernel: [74739.890023] R13: ffff88171d6be150 R14: 0000000000000000 R15: 00007f58eeeee9d0
2014-01-18T13:49:27.868086+00:00 bl-cassoop-p11 kernel: [74739.890024] FS:  00007f58fc73e700(0000) GS:ffff88181fa60000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868087+00:00 bl-cassoop-p11 kernel: [74739.890025] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868089+00:00 bl-cassoop-p11 kernel: [74739.890026] CR2: 00007fd1e9a6b218 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.868090+00:00 bl-cassoop-p11 kernel: [74739.890027] Stack:
2014-01-18T13:49:27.868091+00:00 bl-cassoop-p11 kernel: [74739.890028]  ffff8817f927de20 ffffffff8174787e ffff8817f927dea0 ffffffff81063fda
2014-01-18T13:49:27.868093+00:00 bl-cassoop-p11 kernel: [74739.890031]  fffffffffffffff5 0000000000010000 ffff88171d6be0d0 00000000a35ce200
2014-01-18T13:49:27.868097+00:00 bl-cassoop-p11 kernel: [74739.890034]  00007f58eeeedff0 ffff8817f7bd2400 ffffffff8167e2e0 00000000f927df58
2014-01-18T13:49:27.868098+00:00 bl-cassoop-p11 kernel: [74739.890037] Call Trace:
2014-01-18T13:49:27.868099+00:00 bl-cassoop-p11 kernel: [74739.890046]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.868101+00:00 bl-cassoop-p11 kernel: [74739.890048]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:27.868103+00:00 bl-cassoop-p11 kernel: [74739.890051]  [<ffffffff8167e2e0>] ? do_tcp_setsockopt.isra.29.part.30+0x60/0x720
2014-01-18T13:49:27.868104+00:00 bl-cassoop-p11 kernel: [74739.890053]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:27.868106+00:00 bl-cassoop-p11 kernel: [74739.890055]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:27.868107+00:00 bl-cassoop-p11 kernel: [74739.890057]  [<ffffffff811c4ae8>] ? setfl+0x118/0x170
2014-01-18T13:49:27.868111+00:00 bl-cassoop-p11 kernel: [74739.890059]  [<ffffffff811c5038>] ? do_fcntl+0x258/0x340
2014-01-18T13:49:27.868113+00:00 bl-cassoop-p11 kernel: [74739.890061]  [<ffffffff8167eab3>] ? tcp_setsockopt+0x33/0x40
2014-01-18T13:49:27.868114+00:00 bl-cassoop-p11 kernel: [74739.890063]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:27.868116+00:00 bl-cassoop-p11 kernel: [74739.890066]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:27.868117+00:00 bl-cassoop-p11 kernel: [74739.890076]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.868120+00:00 bl-cassoop-p11 kernel: [74739.890081] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.868121+00:00 bl-cassoop-p11 kernel: [74739.890139] NMI backtrace for cpu 7
2014-01-18T13:49:27.868125+00:00 bl-cassoop-p11 kernel: [74739.890143] CPU: 7 PID: 0 Comm: swapper/7 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868127+00:00 bl-cassoop-p11 kernel: [74739.890144] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868129+00:00 bl-cassoop-p11 kernel: [74739.890145] task: ffff8817fad84650 ti: ffff8817fad8e000 task.ti: ffff8817fad8e000
2014-01-18T13:49:27.868131+00:00 bl-cassoop-p11 kernel: [74739.890147] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.868132+00:00 bl-cassoop-p11 kernel: [74739.890150] RSP: 0018:ffff8817fad8fda8  EFLAGS: 00000046
2014-01-18T13:49:27.868134+00:00 bl-cassoop-p11 kernel: [74739.890151] RAX: 0000000000000001 RBX: 0000000000000002 RCX: 0000000000000001
2014-01-18T13:49:27.868135+00:00 bl-cassoop-p11 kernel: [74739.890152] RDX: 0000000000000000 RSI: ffff8817fad8ffd8 RDI: 0000000000000007
2014-01-18T13:49:27.868139+00:00 bl-cassoop-p11 kernel: [74739.890153] RBP: ffff8817fad8fdd8 R08: 0000000000006145 R09: 0000000008d33840
2014-01-18T13:49:27.868140+00:00 bl-cassoop-p11 kernel: [74739.890154] R10: 0000000000001d1b R11: 0000000000000000 R12: 0000000000000002
2014-01-18T13:49:27.868142+00:00 bl-cassoop-p11 kernel: [74739.890155] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000002
2014-01-18T13:49:27.868143+00:00 bl-cassoop-p11 kernel: [74739.890157] FS:  0000000000000000(0000) GS:ffff88301f260000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868145+00:00 bl-cassoop-p11 kernel: [74739.890158] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868146+00:00 bl-cassoop-p11 kernel: [74739.890159] CR2: 0000000001ed5cc0 CR3: 000000013fb8b000 CR4: 00000000000407e0
2014-01-18T13:49:27.868147+00:00 bl-cassoop-p11 kernel: [74739.890160] Stack:
2014-01-18T13:49:27.868149+00:00 bl-cassoop-p11 kernel: [74739.890161]  ffff8817fad8fdd8 00000007810bc824 ffffe8ffffa60320 000043f36565e67b
2014-01-18T13:49:27.868152+00:00 bl-cassoop-p11 kernel: [74739.890164]  ffffffff81c8f928 ffffffff81c8f860 ffff8817fad8fe38 ffffffff815e5a9f
2014-01-18T13:49:27.868154+00:00 bl-cassoop-p11 kernel: [74739.890167]  0000000000000000 0000000000324c12 0000000000000000 0000000000324c12
2014-01-18T13:49:27.868155+00:00 bl-cassoop-p11 kernel: [74739.890170] Call Trace:
2014-01-18T13:49:27.868157+00:00 bl-cassoop-p11 kernel: [74739.890173]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.868158+00:00 bl-cassoop-p11 kernel: [74739.890175]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.868160+00:00 bl-cassoop-p11 kernel: [74739.890177]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.868161+00:00 bl-cassoop-p11 kernel: [74739.890180]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.868166+00:00 bl-cassoop-p11 kernel: [74739.890182]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.868168+00:00 bl-cassoop-p11 kernel: [74739.890185]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.868214+00:00 bl-cassoop-p11 kernel: [74739.890186] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.868215+00:00 bl-cassoop-p11 kernel: [74739.890206] NMI backtrace for cpu 8
2014-01-18T13:49:27.868217+00:00 bl-cassoop-p11 kernel: [74739.890213] CPU: 8 PID: 42507 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868218+00:00 bl-cassoop-p11 kernel: [74739.890219] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868220+00:00 bl-cassoop-p11 kernel: [74739.890225] task: ffff881176ee4650 ti: ffff880016e20000 task.ti: ffff880016e20000
2014-01-18T13:49:27.868225+00:00 bl-cassoop-p11 kernel: [74739.890229] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.868227+00:00 bl-cassoop-p11 kernel: [74739.890239] RSP: 0018:ffff880016e21e68  EFLAGS: 00000087
2014-01-18T13:49:27.868228+00:00 bl-cassoop-p11 kernel: [74739.890245] RAX: ffff880016e21ea0 RBX: ffff881176ee4650 RCX: 0000000000000001
2014-01-18T13:49:27.868230+00:00 bl-cassoop-p11 kernel: [74739.890250] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.868231+00:00 bl-cassoop-p11 kernel: [74739.890255] RBP: ffff880016e21e68 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.868233+00:00 bl-cassoop-p11 kernel: [74739.890261] R10: 0000000000000001 R11: 0000000000000400 R12: 0000000000000000
2014-01-18T13:49:27.868234+00:00 bl-cassoop-p11 kernel: [74739.890267] R13: ffff881176ee4650 R14: 0000000000000000 R15: 0000000000000003
2014-01-18T13:49:27.868236+00:00 bl-cassoop-p11 kernel: [74739.890270] FS:  00007f58eeeee700(0000) GS:ffff88181fa80000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868239+00:00 bl-cassoop-p11 kernel: [74739.890271] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868241+00:00 bl-cassoop-p11 kernel: [74739.890272] CR2: 000000078be993a8 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.868242+00:00 bl-cassoop-p11 kernel: [74739.890273] Stack:
2014-01-18T13:49:27.868243+00:00 bl-cassoop-p11 kernel: [74739.890274]  ffff880016e21e78 ffffffff8174787e ffff880016e21ee8 ffffffff810676e5
2014-01-18T13:49:27.868245+00:00 bl-cassoop-p11 kernel: [74739.890277]  ffff880016e21f08 ffff880016e21ea0 ffff881176ee4650 ffff880016e21ea0
2014-01-18T13:49:27.868246+00:00 bl-cassoop-p11 kernel: [74739.890280]  ffff880016e21ea0 0000000100000000 0001e00200000001 ffff881176ee4650
2014-01-18T13:49:27.868247+00:00 bl-cassoop-p11 kernel: [74739.890283] Call Trace:
2014-01-18T13:49:27.868252+00:00 bl-cassoop-p11 kernel: [74739.890285]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.868253+00:00 bl-cassoop-p11 kernel: [74739.890287]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.868255+00:00 bl-cassoop-p11 kernel: [74739.890290]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.868256+00:00 bl-cassoop-p11 kernel: [74739.890292]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.868258+00:00 bl-cassoop-p11 kernel: [74739.890294]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:27.868260+00:00 bl-cassoop-p11 kernel: [74739.890297]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.868262+00:00 bl-cassoop-p11 kernel: [74739.890298] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.868266+00:00 bl-cassoop-p11 kernel: [74739.890319] NMI backtrace for cpu 9
2014-01-18T13:49:27.868268+00:00 bl-cassoop-p11 kernel: [74739.890323] CPU: 9 PID: 0 Comm: swapper/9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868270+00:00 bl-cassoop-p11 kernel: [74739.890324] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868271+00:00 bl-cassoop-p11 kernel: [74739.890326] task: ffff8817fad98000 ti: ffff8817fad92000 task.ti: ffff8817fad92000
2014-01-18T13:49:27.868273+00:00 bl-cassoop-p11 kernel: [74739.890327] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.868274+00:00 bl-cassoop-p11 kernel: [74739.890330] RSP: 0018:ffff8817fad93da8  EFLAGS: 00000046
2014-01-18T13:49:27.868276+00:00 bl-cassoop-p11 kernel: [74739.890332] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.868277+00:00 bl-cassoop-p11 kernel: [74739.890333] RDX: 0000000000000000 RSI: ffff8817fad93fd8 RDI: 0000000000000009
2014-01-18T13:49:27.868282+00:00 bl-cassoop-p11 kernel: [74739.890334] RBP: ffff8817fad93dd8 R08: 00000000000062ad R09: 000000000342e4d9
2014-01-18T13:49:27.868283+00:00 bl-cassoop-p11 kernel: [74739.890335] R10: 0000000000001b7b R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.868285+00:00 bl-cassoop-p11 kernel: [74739.890336] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.868286+00:00 bl-cassoop-p11 kernel: [74739.890337] FS:  0000000000000000(0000) GS:ffff88301f280000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868287+00:00 bl-cassoop-p11 kernel: [74739.890339] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868289+00:00 bl-cassoop-p11 kernel: [74739.890340] CR2: 00007f54cc04ff90 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.868290+00:00 bl-cassoop-p11 kernel: [74739.890341] Stack:
2014-01-18T13:49:27.868294+00:00 bl-cassoop-p11 kernel: [74739.890341]  ffff8817fad93dd8 00000009810bc824 ffffe8ffffa80320 000043f364e92319
2014-01-18T13:49:27.868296+00:00 bl-cassoop-p11 kernel: [74739.890345]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad93e38 ffffffff815e5a9f
2014-01-18T13:49:27.868297+00:00 bl-cassoop-p11 kernel: [74739.890348]  0000000000000000 000000001c1d6b6e 0000000000000000 000000001c1d6b6e
2014-01-18T13:49:27.868298+00:00 bl-cassoop-p11 kernel: [74739.890365] Call Trace:
2014-01-18T13:49:27.868300+00:00 bl-cassoop-p11 kernel: [74739.890382]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.868301+00:00 bl-cassoop-p11 kernel: [74739.890399]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.868303+00:00 bl-cassoop-p11 kernel: [74739.890416]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.868304+00:00 bl-cassoop-p11 kernel: [74739.890433]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.868354+00:00 bl-cassoop-p11 kernel: [74739.890447]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.868356+00:00 bl-cassoop-p11 kernel: [74739.890450]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.868359+00:00 bl-cassoop-p11 kernel: [74739.890451] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.868360+00:00 bl-cassoop-p11 kernel: [74739.890472] NMI backtrace for cpu 10
2014-01-18T13:49:27.868361+00:00 bl-cassoop-p11 kernel: [74739.890474] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868363+00:00 bl-cassoop-p11 kernel: [74739.890475] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868365+00:00 bl-cassoop-p11 kernel: [74739.890476] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:49:27.868373+00:00 bl-cassoop-p11 kernel: [74739.890477] RIP: 0010:[<ffffffff8101b436>]  [<ffffffff8101b436>] native_read_tsc+0x6/0x20
2014-01-18T13:49:27.868375+00:00 bl-cassoop-p11 kernel: [74739.890480] RSP: 0000:ffff88181faa3c88  EFLAGS: 00000046
2014-01-18T13:49:27.868376+00:00 bl-cassoop-p11 kernel: [74739.890482] RAX: 000000003c379f44 RBX: 000000000000000a RCX: 000000003c379f13
2014-01-18T13:49:27.868378+00:00 bl-cassoop-p11 kernel: [74739.890483] RDX: 000000000000b118 RSI: 0000000000000100 RDI: 000000000003ef90
2014-01-18T13:49:27.868379+00:00 bl-cassoop-p11 kernel: [74739.890484] RBP: ffff88181faa3c88 R08: ffffffff81d03da0 R09: 0000000000000000
2014-01-18T13:49:27.868381+00:00 bl-cassoop-p11 kernel: [74739.890485] R10: 0000000000001719 R11: 0000000000001718 R12: 000000003c379ef5
2014-01-18T13:49:27.868382+00:00 bl-cassoop-p11 kernel: [74739.890486] R13: 000000000000000a R14: 000000000003ef90 R15: 0000000000000092
2014-01-18T13:49:27.868386+00:00 bl-cassoop-p11 kernel: [74739.890487] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868388+00:00 bl-cassoop-p11 kernel: [74739.890488] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868389+00:00 bl-cassoop-p11 kernel: [74739.890489] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:27.868390+00:00 bl-cassoop-p11 kernel: [74739.890490] Stack:
2014-01-18T13:49:27.868391+00:00 bl-cassoop-p11 kernel: [74739.890491]  ffff88181faa3cb8 ffffffff8138038a 00000000000003e9 0000000000001000
2014-01-18T13:49:27.868393+00:00 bl-cassoop-p11 kernel: [74739.890494]  ffffffff81d03da0 0000000000000400 ffff88181faa3cc8 ffffffff813802bc
2014-01-18T13:49:27.868394+00:00 bl-cassoop-p11 kernel: [74739.890497]  ffff88181faa3ce8 ffffffff8104383a 0000000000000002 000000000000b06a
2014-01-18T13:49:27.868395+00:00 bl-cassoop-p11 kernel: [74739.890499] Call Trace:
2014-01-18T13:49:27.868399+00:00 bl-cassoop-p11 kernel: [74739.890500]  <IRQ>
2014-01-18T13:49:27.868400+00:00 bl-cassoop-p11 kernel: [74739.890501]  [<ffffffff8138038a>] delay_tsc+0x4a/0x80
2014-01-18T13:49:27.868402+00:00 bl-cassoop-p11 kernel: [74739.890512]  [<ffffffff813802bc>] __const_udelay+0x2c/0x30
2014-01-18T13:49:27.868404+00:00 bl-cassoop-p11 kernel: [74739.890514]  [<ffffffff8104383a>] native_safe_apic_wait_icr_idle+0x2a/0x60
2014-01-18T13:49:27.868405+00:00 bl-cassoop-p11 kernel: [74739.890516]  [<ffffffff8104529f>] default_send_IPI_mask_sequence_phys+0xcf/0xe0
2014-01-18T13:49:27.868407+00:00 bl-cassoop-p11 kernel: [74739.890520]  [<ffffffff81049a57>] physflat_send_IPI_all+0x17/0x20
2014-01-18T13:49:27.868409+00:00 bl-cassoop-p11 kernel: [74739.890522]  [<ffffffff81045431>] arch_trigger_all_cpu_backtrace+0x61/0xa0
2014-01-18T13:49:27.868413+00:00 bl-cassoop-p11 kernel: [74739.890524]  [<ffffffff81105b04>] print_cpu_stall+0xd4/0x140
2014-01-18T13:49:27.868414+00:00 bl-cassoop-p11 kernel: [74739.890527]  [<ffffffff81106507>] __rcu_pending+0x77/0x1d0
2014-01-18T13:49:27.868416+00:00 bl-cassoop-p11 kernel: [74739.890529]  [<ffffffff81107e4d>] rcu_check_callbacks+0xed/0x1b0
2014-01-18T13:49:27.868418+00:00 bl-cassoop-p11 kernel: [74739.890532]  [<ffffffff810741b8>] update_process_times+0x48/0x80
2014-01-18T13:49:27.868419+00:00 bl-cassoop-p11 kernel: [74739.890534]  [<ffffffff810c59e3>] tick_sched_handle.isra.10+0x33/0x70
2014-01-18T13:49:27.868421+00:00 bl-cassoop-p11 kernel: [74739.890537]  [<ffffffff810c5b0c>] tick_sched_timer+0x4c/0x80
2014-01-18T13:49:27.868422+00:00 bl-cassoop-p11 kernel: [74739.890546]  [<ffffffff8108c8b8>] __run_hrtimer+0x78/0x230
2014-01-18T13:49:27.868424+00:00 bl-cassoop-p11 kernel: [74739.890557]  [<ffffffff810c5ac0>] ? tick_nohz_handler+0xa0/0xa0
2014-01-18T13:49:27.868428+00:00 bl-cassoop-p11 kernel: [74739.890567]  [<ffffffff8108d0bf>] hrtimer_interrupt+0xff/0x240
2014-01-18T13:49:27.868430+00:00 bl-cassoop-p11 kernel: [74739.890576]  [<ffffffff810437bb>] local_apic_timer_interrupt+0x3b/0x60
2014-01-18T13:49:27.868432+00:00 bl-cassoop-p11 kernel: [74739.890587]  [<ffffffff81752d25>] smp_apic_timer_interrupt+0x45/0x60
2014-01-18T13:49:27.868434+00:00 bl-cassoop-p11 kernel: [74739.890597]  [<ffffffff8175169d>] apic_timer_interrupt+0x6d/0x80
2014-01-18T13:49:27.868435+00:00 bl-cassoop-p11 kernel: [74739.890597]  <EOI>
2014-01-18T13:49:27.868436+00:00 bl-cassoop-p11 kernel: [74739.890599]  [<ffffffff81731068>] ? oom_kill_process.part.6+0xae/0x2cf
2014-01-18T13:49:27.868438+00:00 bl-cassoop-p11 kernel: [74739.890603]  [<ffffffff8173105d>] ? oom_kill_process.part.6+0xa3/0x2cf
2014-01-18T13:49:27.868442+00:00 bl-cassoop-p11 kernel: [74739.890606]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:49:27.868444+00:00 bl-cassoop-p11 kernel: [74739.890608]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:49:27.868445+00:00 bl-cassoop-p11 kernel: [74739.890610]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:49:27.868447+00:00 bl-cassoop-p11 kernel: [74739.890612]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:49:27.868449+00:00 bl-cassoop-p11 kernel: [74739.890614]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:49:27.868450+00:00 bl-cassoop-p11 kernel: [74739.890617]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:49:27.868452+00:00 bl-cassoop-p11 kernel: [74739.890620]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.868454+00:00 bl-cassoop-p11 kernel: [74739.890622]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.868459+00:00 bl-cassoop-p11 kernel: [74739.890625]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:27.868461+00:00 bl-cassoop-p11 kernel: [74739.890627]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:27.868505+00:00 bl-cassoop-p11 kernel: [74739.890629]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:27.868507+00:00 bl-cassoop-p11 kernel: [74739.890632]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.868509+00:00 bl-cassoop-p11 kernel: [74739.890634]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:27.868510+00:00 bl-cassoop-p11 kernel: [74739.890636]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.868512+00:00 bl-cassoop-p11 kernel: [74739.890639]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:49:27.868516+00:00 bl-cassoop-p11 kernel: [74739.890641]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.868518+00:00 bl-cassoop-p11 kernel: [74739.890643]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.868519+00:00 bl-cassoop-p11 kernel: [74739.890646]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:27.868521+00:00 bl-cassoop-p11 kernel: [74739.890648]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:27.868522+00:00 bl-cassoop-p11 kernel: [74739.890650]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.868524+00:00 bl-cassoop-p11 kernel: [74739.890652]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:27.868525+00:00 bl-cassoop-p11 kernel: [74739.890654]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.868527+00:00 bl-cassoop-p11 kernel: [74739.890657]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.868532+00:00 bl-cassoop-p11 kernel: [74739.890658] Code: e8 4c 8b 65 f0 4c 8b 6d f8 c9 c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 5d c3 90 90 90 90 90 55 48 89 e5 0f 31 <89> c1 48 89 d0 48 c1 e0 20 48 09 c8 5d c3 66 66 66 2e 0f 1f 84
2014-01-18T13:49:27.868533+00:00 bl-cassoop-p11 kernel: [74739.890683] NMI backtrace for cpu 11
2014-01-18T13:49:27.868535+00:00 bl-cassoop-p11 kernel: [74739.890695] CPU: 11 PID: 0 Comm: swapper/11 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868537+00:00 bl-cassoop-p11 kernel: [74739.890706] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868538+00:00 bl-cassoop-p11 kernel: [74739.890716] task: ffff8817fad9aee0 ti: ffff8817fad96000 task.ti: ffff8817fad96000
2014-01-18T13:49:27.868540+00:00 bl-cassoop-p11 kernel: [74739.890725] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.868542+00:00 bl-cassoop-p11 kernel: [74739.890743] RSP: 0018:ffff8817fad97da8  EFLAGS: 00000046
2014-01-18T13:49:27.868546+00:00 bl-cassoop-p11 kernel: [74739.890752] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.868548+00:00 bl-cassoop-p11 kernel: [74739.890762] RDX: 0000000000000000 RSI: ffff8817fad97fd8 RDI: 000000000000000b
2014-01-18T13:49:27.868549+00:00 bl-cassoop-p11 kernel: [74739.890772] RBP: ffff8817fad97dd8 R08: 00000000000244ed R09: 00000000ca9fd781
2014-01-18T13:49:27.868551+00:00 bl-cassoop-p11 kernel: [74739.890782] R10: 0000000000001c4a R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.868552+00:00 bl-cassoop-p11 kernel: [74739.890791] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.868553+00:00 bl-cassoop-p11 kernel: [74739.890799] FS:  0000000000000000(0000) GS:ffff88301f2a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868555+00:00 bl-cassoop-p11 kernel: [74739.890800] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868559+00:00 bl-cassoop-p11 kernel: [74739.890802] CR2: 0000000001bc3000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.868560+00:00 bl-cassoop-p11 kernel: [74739.890803] Stack:
2014-01-18T13:49:27.868561+00:00 bl-cassoop-p11 kernel: [74739.890804]  ffff8817fad97dd8 0000000b810bc824 ffffe8ffffaa0320 000043f364ea178b
2014-01-18T13:49:27.868563+00:00 bl-cassoop-p11 kernel: [74739.890809]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fad97e38 ffffffff815e5a9f
2014-01-18T13:49:27.868564+00:00 bl-cassoop-p11 kernel: [74739.890813]  0000000000000000 000000001dd4d15f 0000000000000000 000000001dd4d15f
2014-01-18T13:49:27.868565+00:00 bl-cassoop-p11 kernel: [74739.890817] Call Trace:
2014-01-18T13:49:27.868567+00:00 bl-cassoop-p11 kernel: [74739.890821]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.868569+00:00 bl-cassoop-p11 kernel: [74739.890824]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.868573+00:00 bl-cassoop-p11 kernel: [74739.890827]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.868575+00:00 bl-cassoop-p11 kernel: [74739.890830]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.868576+00:00 bl-cassoop-p11 kernel: [74739.890833]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.868578+00:00 bl-cassoop-p11 kernel: [74739.890837]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.868580+00:00 bl-cassoop-p11 kernel: [74739.890838] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.868582+00:00 bl-cassoop-p11 kernel: [74739.890868] NMI backtrace for cpu 13
2014-01-18T13:49:27.868584+00:00 bl-cassoop-p11 kernel: [74739.890869] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868588+00:00 bl-cassoop-p11 kernel: [74739.890870] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868589+00:00 bl-cassoop-p11 kernel: [74739.890872] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:49:27.868591+00:00 bl-cassoop-p11 kernel: [74739.890873] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:27.868593+00:00 bl-cassoop-p11 kernel: [74739.890875] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:49:27.868594+00:00 bl-cassoop-p11 kernel: [74739.890876] RAX: 0000000000000002 RBX: ffff88301f2d5088 RCX: ffff88181fa386a8
2014-01-18T13:49:27.868596+00:00 bl-cassoop-p11 kernel: [74739.890877] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.868597+00:00 bl-cassoop-p11 kernel: [74739.890878] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:49:27.868601+00:00 bl-cassoop-p11 kernel: [74739.890879] R10: 0000000000000004 R11: 0000000000000000 R12: ffff88301f2d5080
2014-01-18T13:49:27.868603+00:00 bl-cassoop-p11 kernel: [74739.890880] R13: ffffffff8105ae90 R14: ffff882ff8363820 R15: 0000000000015040
2014-01-18T13:49:27.868650+00:00 bl-cassoop-p11 kernel: [74739.890881] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868651+00:00 bl-cassoop-p11 kernel: [74739.890882] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868653+00:00 bl-cassoop-p11 kernel: [74739.890883] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.868654+00:00 bl-cassoop-p11 kernel: [74739.890884] Stack:
2014-01-18T13:49:27.868655+00:00 bl-cassoop-p11 kernel: [74739.890884]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:49:27.868657+00:00 bl-cassoop-p11 kernel: [74739.890887]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:49:27.868661+00:00 bl-cassoop-p11 kernel: [74739.890890]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:49:27.868662+00:00 bl-cassoop-p11 kernel: [74739.890893] Call Trace:
2014-01-18T13:49:27.868664+00:00 bl-cassoop-p11 kernel: [74739.890895]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.868665+00:00 bl-cassoop-p11 kernel: [74739.890898]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.868667+00:00 bl-cassoop-p11 kernel: [74739.890900]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.868669+00:00 bl-cassoop-p11 kernel: [74739.890903]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:27.868670+00:00 bl-cassoop-p11 kernel: [74739.890905]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:27.868675+00:00 bl-cassoop-p11 kernel: [74739.890922]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:27.868677+00:00 bl-cassoop-p11 kernel: [74739.890939]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:27.868678+00:00 bl-cassoop-p11 kernel: [74739.890956]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:27.868680+00:00 bl-cassoop-p11 kernel: [74739.890972]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.868681+00:00 bl-cassoop-p11 kernel: [74739.890989]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.868683+00:00 bl-cassoop-p11 kernel: [74739.891004]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.868685+00:00 bl-cassoop-p11 kernel: [74739.891006]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.868686+00:00 bl-cassoop-p11 kernel: [74739.891009]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.868691+00:00 bl-cassoop-p11 kernel: [74739.891011]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:27.868692+00:00 bl-cassoop-p11 kernel: [74739.891013]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:27.868694+00:00 bl-cassoop-p11 kernel: [74739.891016]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:27.868695+00:00 bl-cassoop-p11 kernel: [74739.891018]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:27.868697+00:00 bl-cassoop-p11 kernel: [74739.891020]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:27.868698+00:00 bl-cassoop-p11 kernel: [74739.891023]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.868700+00:00 bl-cassoop-p11 kernel: [74739.891025]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.868704+00:00 bl-cassoop-p11 kernel: [74739.891028]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.868706+00:00 bl-cassoop-p11 kernel: [74739.891030]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.868709+00:00 bl-cassoop-p11 kernel: [74739.891031] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:27.868710+00:00 bl-cassoop-p11 kernel: [74739.891051] NMI backtrace for cpu 12
2014-01-18T13:49:27.868712+00:00 bl-cassoop-p11 kernel: [74739.891054] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868713+00:00 bl-cassoop-p11 kernel: [74739.891055] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868715+00:00 bl-cassoop-p11 kernel: [74739.891056] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:49:27.868719+00:00 bl-cassoop-p11 kernel: [74739.891057] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:27.868721+00:00 bl-cassoop-p11 kernel: [74739.891060] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:49:27.868722+00:00 bl-cassoop-p11 kernel: [74739.891061] RAX: 0000000000000002 RBX: ffff88181fad5088 RCX: ffff88181fa38680
2014-01-18T13:49:27.868724+00:00 bl-cassoop-p11 kernel: [74739.891062] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.868725+00:00 bl-cassoop-p11 kernel: [74739.891063] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:49:27.868727+00:00 bl-cassoop-p11 kernel: [74739.891064] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff88181fad5080
2014-01-18T13:49:27.868728+00:00 bl-cassoop-p11 kernel: [74739.891065] R13: ffffffff8105ae90 R14: ffff8817f8e45a10 R15: 0000000000015040
2014-01-18T13:49:27.868733+00:00 bl-cassoop-p11 kernel: [74739.891067] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868734+00:00 bl-cassoop-p11 kernel: [74739.891068] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868736+00:00 bl-cassoop-p11 kernel: [74739.891069] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.868737+00:00 bl-cassoop-p11 kernel: [74739.891070] Stack:
2014-01-18T13:49:27.868738+00:00 bl-cassoop-p11 kernel: [74739.891070]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:49:27.868740+00:00 bl-cassoop-p11 kernel: [74739.891074]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:49:27.868741+00:00 bl-cassoop-p11 kernel: [74739.891077]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:49:27.868742+00:00 bl-cassoop-p11 kernel: [74739.891079] Call Trace:
2014-01-18T13:49:27.868747+00:00 bl-cassoop-p11 kernel: [74739.891083]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.868749+00:00 bl-cassoop-p11 kernel: [74739.891085]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.868751+00:00 bl-cassoop-p11 kernel: [74739.891087]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.868752+00:00 bl-cassoop-p11 kernel: [74739.891090]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:27.868799+00:00 bl-cassoop-p11 kernel: [74739.891092]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:27.868800+00:00 bl-cassoop-p11 kernel: [74739.891095]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:27.868802+00:00 bl-cassoop-p11 kernel: [74739.891097]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:27.868806+00:00 bl-cassoop-p11 kernel: [74739.891099]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:27.868808+00:00 bl-cassoop-p11 kernel: [74739.891101]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:27.868809+00:00 bl-cassoop-p11 kernel: [74739.891111]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:27.868811+00:00 bl-cassoop-p11 kernel: [74739.891122]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:27.868813+00:00 bl-cassoop-p11 kernel: [74739.891132]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:27.868814+00:00 bl-cassoop-p11 kernel: [74739.891142]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:27.868816+00:00 bl-cassoop-p11 kernel: [74739.891153]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:49:27.868818+00:00 bl-cassoop-p11 kernel: [74739.891162]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.868822+00:00 bl-cassoop-p11 kernel: [74739.891165]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.868823+00:00 bl-cassoop-p11 kernel: [74739.891167]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.868825+00:00 bl-cassoop-p11 kernel: [74739.891170]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.868827+00:00 bl-cassoop-p11 kernel: [74739.891171] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:27.868829+00:00 bl-cassoop-p11 kernel: [74739.891191] NMI backtrace for cpu 14
2014-01-18T13:49:27.868830+00:00 bl-cassoop-p11 kernel: [74739.891193] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.868832+00:00 bl-cassoop-p11 kernel: [74739.891195] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.868836+00:00 bl-cassoop-p11 kernel: [74739.891197] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:49:27.868838+00:00 bl-cassoop-p11 kernel: [74739.891198] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:27.868840+00:00 bl-cassoop-p11 kernel: [74739.891201] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:49:27.868841+00:00 bl-cassoop-p11 kernel: [74739.891203] RAX: 0000000000000008 RBX: ffff88181faf5088 RCX: ffff88181fa986d0
2014-01-18T13:49:27.868843+00:00 bl-cassoop-p11 kernel: [74739.891205] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.868844+00:00 bl-cassoop-p11 kernel: [74739.891206] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:49:27.868846+00:00 bl-cassoop-p11 kernel: [74739.891208] R10: 0000000000000004 R11: 0000000000000058 R12: ffff88181faf5080
2014-01-18T13:49:27.868850+00:00 bl-cassoop-p11 kernel: [74739.891209] R13: ffffffff8105ae90 R14: ffff88195e365180 R15: 0000000000015040
2014-01-18T13:49:27.868852+00:00 bl-cassoop-p11 kernel: [74739.891211] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.868853+00:00 bl-cassoop-p11 kernel: [74739.891213] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.868854+00:00 bl-cassoop-p11 kernel: [74739.891215] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:49:27.868855+00:00 bl-cassoop-p11 kernel: [74739.891216] Stack:
2014-01-18T13:49:27.868857+00:00 bl-cassoop-p11 kernel: [74739.891217]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:49:27.868858+00:00 bl-cassoop-p11 kernel: [74739.891222]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:49:27.868860+00:00 bl-cassoop-p11 kernel: [74739.891226]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:49:27.868863+00:00 bl-cassoop-p11 kernel: [74739.891231] Call Trace:
2014-01-18T13:49:27.868865+00:00 bl-cassoop-p11 kernel: [74739.891234]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.868867+00:00 bl-cassoop-p11 kernel: [74739.891238]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.868868+00:00 bl-cassoop-p11 kernel: [74739.891241]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.868870+00:00 bl-cassoop-p11 kernel: [74739.891245]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:27.868871+00:00 bl-cassoop-p11 kernel: [74739.891248]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:27.868873+00:00 bl-cassoop-p11 kernel: [74739.891251]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:27.868877+00:00 bl-cassoop-p11 kernel: [74739.891255]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:27.868879+00:00 bl-cassoop-p11 kernel: [74739.891258]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:27.868880+00:00 bl-cassoop-p11 kernel: [74739.891269]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.868882+00:00 bl-cassoop-p11 kernel: [74739.891281]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.868883+00:00 bl-cassoop-p11 kernel: [74739.891293]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.868885+00:00 bl-cassoop-p11 kernel: [74739.891304]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:27.868886+00:00 bl-cassoop-p11 kernel: [74739.891314]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:27.868888+00:00 bl-cassoop-p11 kernel: [74739.891325]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:27.868892+00:00 bl-cassoop-p11 kernel: [74739.891329]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:27.868894+00:00 bl-cassoop-p11 kernel: [74739.891332]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:27.868896+00:00 bl-cassoop-p11 kernel: [74739.891336]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:27.868897+00:00 bl-cassoop-p11 kernel: [74739.891340]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:27.868899+00:00 bl-cassoop-p11 kernel: [74739.891343]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.868901+00:00 bl-cassoop-p11 kernel: [74739.891347]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.868948+00:00 bl-cassoop-p11 kernel: [74739.891350]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:49:27.868952+00:00 bl-cassoop-p11 kernel: [74739.891354]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:49:27.868954+00:00 bl-cassoop-p11 kernel: [74739.891357]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:49:27.868955+00:00 bl-cassoop-p11 kernel: [74739.891361]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:49:27.868957+00:00 bl-cassoop-p11 kernel: [74739.891365]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:27.868959+00:00 bl-cassoop-p11 kernel: [74739.891368]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:49:27.868960+00:00 bl-cassoop-p11 kernel: [74739.891371]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:27.868962+00:00 bl-cassoop-p11 kernel: [74739.891375]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:49:27.868963+00:00 bl-cassoop-p11 kernel: [74739.891378]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:49:27.868968+00:00 bl-cassoop-p11 kernel: [74739.891381]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:49:27.868970+00:00 bl-cassoop-p11 kernel: [74739.891384]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:49:27.868971+00:00 bl-cassoop-p11 kernel: [74739.891387]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:49:27.868973+00:00 bl-cassoop-p11 kernel: [74739.891390]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:49:27.868974+00:00 bl-cassoop-p11 kernel: [74739.891394]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.868976+00:00 bl-cassoop-p11 kernel: [74739.891397]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:49:27.868977+00:00 bl-cassoop-p11 kernel: [74739.891400]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.868982+00:00 bl-cassoop-p11 kernel: [74739.891403]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:49:27.868983+00:00 bl-cassoop-p11 kernel: [74739.891407]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:49:27.868985+00:00 bl-cassoop-p11 kernel: [74739.891409]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:49:27.868986+00:00 bl-cassoop-p11 kernel: [74739.891413]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:49:27.868988+00:00 bl-cassoop-p11 kernel: [74739.891415]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.868989+00:00 bl-cassoop-p11 kernel: [74739.891419]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.868991+00:00 bl-cassoop-p11 kernel: [74739.891422]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.868992+00:00 bl-cassoop-p11 kernel: [74739.891434]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:49:27.869000+00:00 bl-cassoop-p11 kernel: [74739.891445]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:27.869002+00:00 bl-cassoop-p11 kernel: [74739.891455]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:27.869004+00:00 bl-cassoop-p11 kernel: [74739.891466]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.869005+00:00 bl-cassoop-p11 kernel: [74739.891477]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.869008+00:00 bl-cassoop-p11 kernel: [74739.891483] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:27.869009+00:00 bl-cassoop-p11 kernel: [74739.891521] NMI backtrace for cpu 15
2014-01-18T13:49:27.869011+00:00 bl-cassoop-p11 kernel: [74739.891524] CPU: 15 PID: 36402 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869015+00:00 bl-cassoop-p11 kernel: [74739.891526] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869016+00:00 bl-cassoop-p11 kernel: [74739.891527] task: ffff8818979f5dc0 ti: ffff88252a4ca000 task.ti: ffff88252a4ca000
2014-01-18T13:49:27.869019+00:00 bl-cassoop-p11 kernel: [74739.891528] RIP: 0010:[<ffffffff813817d3>]  [<ffffffff813817d3>] __write_lock_failed+0x13/0x20
2014-01-18T13:49:27.869020+00:00 bl-cassoop-p11 kernel: [74739.891531] RSP: 0018:ffff88252a4cbc88  EFLAGS: 00000087
2014-01-18T13:49:27.869022+00:00 bl-cassoop-p11 kernel: [74739.891532] RAX: ffff88252a4cbcc0 RBX: ffff8818979f5dc0 RCX: 0000000000000001
2014-01-18T13:49:27.869023+00:00 bl-cassoop-p11 kernel: [74739.891533] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff81c06040
2014-01-18T13:49:27.869024+00:00 bl-cassoop-p11 kernel: [74739.891534] RBP: ffff88252a4cbc88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:27.869029+00:00 bl-cassoop-p11 kernel: [74739.891535] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:27.869030+00:00 bl-cassoop-p11 kernel: [74739.891536] R13: ffff8818979f5dc0 R14: 0000000000000000 R15: 0000000000000002
2014-01-18T13:49:27.869032+00:00 bl-cassoop-p11 kernel: [74739.891537] FS:  0000000000000000(0000) GS:ffff88301f2e0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869033+00:00 bl-cassoop-p11 kernel: [74739.891538] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869035+00:00 bl-cassoop-p11 kernel: [74739.891539] CR2: 00000006d89af728 CR3: 0000001885e2a000 CR4: 00000000000407e0
2014-01-18T13:49:27.869036+00:00 bl-cassoop-p11 kernel: [74739.891540] Stack:
2014-01-18T13:49:27.869037+00:00 bl-cassoop-p11 kernel: [74739.891541]  ffff88252a4cbc98 ffffffff8174787e ffff88252a4cbd08 ffffffff810676e5
2014-01-18T13:49:27.869039+00:00 bl-cassoop-p11 kernel: [74739.891544]  ffff88252a4cbd28 ffff88252a4cbcc0 ffff8818979f5dc0 ffff88252a4cbcc0
2014-01-18T13:49:27.869044+00:00 bl-cassoop-p11 kernel: [74739.891547]  ffff88252a4cbcc0 0000000100000000 0001d7ce00000001 ffff8818979f5dc0
2014-01-18T13:49:27.869045+00:00 bl-cassoop-p11 kernel: [74739.891550] Call Trace:
2014-01-18T13:49:27.869046+00:00 bl-cassoop-p11 kernel: [74739.891552]  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:27.869048+00:00 bl-cassoop-p11 kernel: [74739.891555]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:27.869050+00:00 bl-cassoop-p11 kernel: [74739.891557]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:27.869051+00:00 bl-cassoop-p11 kernel: [74739.891559]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:27.869053+00:00 bl-cassoop-p11 kernel: [74739.891562]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:27.869058+00:00 bl-cassoop-p11 kernel: [74739.891564]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:27.869060+00:00 bl-cassoop-p11 kernel: [74739.891566]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:27.869106+00:00 bl-cassoop-p11 kernel: [74739.891569]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:27.869108+00:00 bl-cassoop-p11 kernel: [74739.891571]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:27.869109+00:00 bl-cassoop-p11 kernel: [74739.891574]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:27.869111+00:00 bl-cassoop-p11 kernel: [74739.891577]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:27.869113+00:00 bl-cassoop-p11 kernel: [74739.891579]  [<ffffffff81747c7c>] retint_signal+0x48/0x8c
2014-01-18T13:49:27.869115+00:00 bl-cassoop-p11 kernel: [74739.891580] Code: c3 b8 f2 ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 81 07 00 00 10 00 f3 90 81 3f 00 00 10 00 <75> f6 f0 81 2f 00 00 10 00 75 e6 5d c3 55 48 89 e5 f0 ff 07 f3
2014-01-18T13:49:27.869119+00:00 bl-cassoop-p11 kernel: [74739.891626] NMI backtrace for cpu 16
2014-01-18T13:49:27.869121+00:00 bl-cassoop-p11 kernel: [74739.891634] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869123+00:00 bl-cassoop-p11 kernel: [74739.891640] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869124+00:00 bl-cassoop-p11 kernel: [74739.891647] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:49:27.869126+00:00 bl-cassoop-p11 kernel: [74739.891661] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:49:27.869128+00:00 bl-cassoop-p11 kernel: [74739.891667] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.869132+00:00 bl-cassoop-p11 kernel: [74739.891677] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:49:27.869134+00:00 bl-cassoop-p11 kernel: [74739.891678] RAX: 0000000000000008 RBX: ffff88181fb15088 RCX: ffff88181fa98720
2014-01-18T13:49:27.869135+00:00 bl-cassoop-p11 kernel: [74739.891680] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.869137+00:00 bl-cassoop-p11 kernel: [74739.891681] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:49:27.869138+00:00 bl-cassoop-p11 kernel: [74739.891683] R10: 0000000000000004 R11: 0000000000000210 R12: ffff88181fb15080
2014-01-18T13:49:27.869140+00:00 bl-cassoop-p11 kernel: [74739.891685] R13: ffffffff8105ae90 R14: ffff880075d0b6b0 R15: 0000000000015040
2014-01-18T13:49:27.869141+00:00 bl-cassoop-p11 kernel: [74739.891687] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869143+00:00 bl-cassoop-p11 kernel: [74739.891688] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869147+00:00 bl-cassoop-p11 kernel: [74739.891690] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869148+00:00 bl-cassoop-p11 kernel: [74739.891691] Stack:
2014-01-18T13:49:27.869149+00:00 bl-cassoop-p11 kernel: [74739.891692]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:49:27.869151+00:00 bl-cassoop-p11 kernel: [74739.891697]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:49:27.869152+00:00 bl-cassoop-p11 kernel: [74739.891702]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:49:27.869153+00:00 bl-cassoop-p11 kernel: [74739.891706] Call Trace:
2014-01-18T13:49:27.869155+00:00 bl-cassoop-p11 kernel: [74739.891710]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.869157+00:00 bl-cassoop-p11 kernel: [74739.891713]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.869161+00:00 bl-cassoop-p11 kernel: [74739.891716]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.869162+00:00 bl-cassoop-p11 kernel: [74739.891719]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:49:27.869164+00:00 bl-cassoop-p11 kernel: [74739.891722]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:49:27.869166+00:00 bl-cassoop-p11 kernel: [74739.891726]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:49:27.869167+00:00 bl-cassoop-p11 kernel: [74739.891729]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:49:27.869169+00:00 bl-cassoop-p11 kernel: [74739.891732]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:49:27.869170+00:00 bl-cassoop-p11 kernel: [74739.891735]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:49:27.869174+00:00 bl-cassoop-p11 kernel: [74739.891738]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:49:27.869176+00:00 bl-cassoop-p11 kernel: [74739.891742]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:49:27.869178+00:00 bl-cassoop-p11 kernel: [74739.891746]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:49:27.869180+00:00 bl-cassoop-p11 kernel: [74739.891749]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:49:27.869181+00:00 bl-cassoop-p11 kernel: [74739.891752]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:49:27.869183+00:00 bl-cassoop-p11 kernel: [74739.891756]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:49:27.869184+00:00 bl-cassoop-p11 kernel: [74739.891759]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:49:27.869186+00:00 bl-cassoop-p11 kernel: [74739.891762]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:49:27.869190+00:00 bl-cassoop-p11 kernel: [74739.891765]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:49:27.869192+00:00 bl-cassoop-p11 kernel: [74739.891769]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:49:27.869194+00:00 bl-cassoop-p11 kernel: [74739.891772]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:49:27.869195+00:00 bl-cassoop-p11 kernel: [74739.891775]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:49:27.869197+00:00 bl-cassoop-p11 kernel: [74739.891787]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:49:27.869198+00:00 bl-cassoop-p11 kernel: [74739.891798]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:49:27.869200+00:00 bl-cassoop-p11 kernel: [74739.891810]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:49:27.869204+00:00 bl-cassoop-p11 kernel: [74739.891820]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:49:27.869206+00:00 bl-cassoop-p11 kernel: [74739.891831]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:49:27.869207+00:00 bl-cassoop-p11 kernel: [74739.891840]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.869209+00:00 bl-cassoop-p11 kernel: [74739.891844]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.869253+00:00 bl-cassoop-p11 kernel: [74739.891847]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.869255+00:00 bl-cassoop-p11 kernel: [74739.891850]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.869257+00:00 bl-cassoop-p11 kernel: [74739.891852] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.869261+00:00 bl-cassoop-p11 kernel: [74739.891883] NMI backtrace for cpu 17
2014-01-18T13:49:27.869263+00:00 bl-cassoop-p11 kernel: [74739.891886] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869265+00:00 bl-cassoop-p11 kernel: [74739.891887] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869266+00:00 bl-cassoop-p11 kernel: [74739.891889] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:49:27.869268+00:00 bl-cassoop-p11 kernel: [74739.891890] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:27.869270+00:00 bl-cassoop-p11 kernel: [74739.891894] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:49:27.869271+00:00 bl-cassoop-p11 kernel: [74739.891895] RAX: 0000000000000006 RBX: ffff88301f315088 RCX: ffff88181fa78748
2014-01-18T13:49:27.869275+00:00 bl-cassoop-p11 kernel: [74739.891897] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.869277+00:00 bl-cassoop-p11 kernel: [74739.891899] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:49:27.869278+00:00 bl-cassoop-p11 kernel: [74739.891900] R10: 0000000000000004 R11: 0000000000017960 R12: ffff88301f315080
2014-01-18T13:49:27.869280+00:00 bl-cassoop-p11 kernel: [74739.891902] R13: ffffffff8105ae90 R14: ffff882feccbfa10 R15: 0000000000015040
2014-01-18T13:49:27.869281+00:00 bl-cassoop-p11 kernel: [74739.891904] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869282+00:00 bl-cassoop-p11 kernel: [74739.891905] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869284+00:00 bl-cassoop-p11 kernel: [74739.891907] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:27.869285+00:00 bl-cassoop-p11 kernel: [74739.891908] Stack:
2014-01-18T13:49:27.869289+00:00 bl-cassoop-p11 kernel: [74739.891909]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:49:27.869291+00:00 bl-cassoop-p11 kernel: [74739.891914]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:49:27.869292+00:00 bl-cassoop-p11 kernel: [74739.891919]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:49:27.869293+00:00 bl-cassoop-p11 kernel: [74739.891923] Call Trace:
2014-01-18T13:49:27.869295+00:00 bl-cassoop-p11 kernel: [74739.891927]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.869296+00:00 bl-cassoop-p11 kernel: [74739.891931]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.869298+00:00 bl-cassoop-p11 kernel: [74739.891948]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:27.869302+00:00 bl-cassoop-p11 kernel: [74739.891965]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:27.869304+00:00 bl-cassoop-p11 kernel: [74739.891983]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:27.869305+00:00 bl-cassoop-p11 kernel: [74739.892000]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:27.869307+00:00 bl-cassoop-p11 kernel: [74739.892017]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:27.869308+00:00 bl-cassoop-p11 kernel: [74739.892034]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:27.869310+00:00 bl-cassoop-p11 kernel: [74739.892040]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:27.869311+00:00 bl-cassoop-p11 kernel: [74739.892044]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:27.869313+00:00 bl-cassoop-p11 kernel: [74739.892047]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:27.869317+00:00 bl-cassoop-p11 kernel: [74739.892050]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:27.869319+00:00 bl-cassoop-p11 kernel: [74739.892053]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:27.869320+00:00 bl-cassoop-p11 kernel: [74739.892056]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:27.869322+00:00 bl-cassoop-p11 kernel: [74739.892059]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.869323+00:00 bl-cassoop-p11 kernel: [74739.892062]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.869325+00:00 bl-cassoop-p11 kernel: [74739.892065]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:49:27.869326+00:00 bl-cassoop-p11 kernel: [74739.892068]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:49:27.869332+00:00 bl-cassoop-p11 kernel: [74739.892072]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:49:27.869334+00:00 bl-cassoop-p11 kernel: [74739.892075]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:49:27.869335+00:00 bl-cassoop-p11 kernel: [74739.892078]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:49:27.869337+00:00 bl-cassoop-p11 kernel: [74739.892081]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.869338+00:00 bl-cassoop-p11 kernel: [74739.892084]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.869341+00:00 bl-cassoop-p11 kernel: [74739.892086] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:27.869342+00:00 bl-cassoop-p11 kernel: [74739.892118] NMI backtrace for cpu 18
2014-01-18T13:49:27.869344+00:00 bl-cassoop-p11 kernel: [74739.892121] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.037 msecs
2014-01-18T13:49:27.869348+00:00 bl-cassoop-p11 kernel: [74739.892125] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869350+00:00 bl-cassoop-p11 kernel: [74739.892127] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869351+00:00 bl-cassoop-p11 kernel: [74739.892128] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:49:27.869398+00:00 bl-cassoop-p11 kernel: [74739.892130] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:27.869399+00:00 bl-cassoop-p11 kernel: [74739.892133] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:49:27.869401+00:00 bl-cassoop-p11 kernel: [74739.892134] RAX: 0000000000000282 RBX: ffff882ff8361750 RCX: 0000000000000001
2014-01-18T13:49:27.869402+00:00 bl-cassoop-p11 kernel: [74739.892136] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:49:27.869406+00:00 bl-cassoop-p11 kernel: [74739.892138] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:49:27.869408+00:00 bl-cassoop-p11 kernel: [74739.892140] R10: 0000000000000001 R11: 0000000000000400 R12: 0000000000000002
2014-01-18T13:49:27.869409+00:00 bl-cassoop-p11 kernel: [74739.892142] R13: ffff88181fa35040 R14: ffff88181fa35050 R15: 0000000000000001
2014-01-18T13:49:27.869411+00:00 bl-cassoop-p11 kernel: [74739.892144] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869412+00:00 bl-cassoop-p11 kernel: [74739.892145] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869413+00:00 bl-cassoop-p11 kernel: [74739.892147] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869414+00:00 bl-cassoop-p11 kernel: [74739.892148] Stack:
2014-01-18T13:49:27.869419+00:00 bl-cassoop-p11 kernel: [74739.892149]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:49:27.869420+00:00 bl-cassoop-p11 kernel: [74739.892154]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:49:27.869422+00:00 bl-cassoop-p11 kernel: [74739.892159]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:49:27.869423+00:00 bl-cassoop-p11 kernel: [74739.892163] Call Trace:
2014-01-18T13:49:27.869424+00:00 bl-cassoop-p11 kernel: [74739.892167]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.869426+00:00 bl-cassoop-p11 kernel: [74739.892169]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.869428+00:00 bl-cassoop-p11 kernel: [74739.892173]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.869429+00:00 bl-cassoop-p11 kernel: [74739.892176]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.869433+00:00 bl-cassoop-p11 kernel: [74739.892179]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.869435+00:00 bl-cassoop-p11 kernel: [74739.892187]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.869437+00:00 bl-cassoop-p11 kernel: [74739.892199]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.869438+00:00 bl-cassoop-p11 kernel: [74739.892210]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.869440+00:00 bl-cassoop-p11 kernel: [74739.892222]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:27.869441+00:00 bl-cassoop-p11 kernel: [74739.892233]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:27.869443+00:00 bl-cassoop-p11 kernel: [74739.892244]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:49:27.869447+00:00 bl-cassoop-p11 kernel: [74739.892252]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:27.869449+00:00 bl-cassoop-p11 kernel: [74739.892255]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:27.869451+00:00 bl-cassoop-p11 kernel: [74739.892259]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:27.869452+00:00 bl-cassoop-p11 kernel: [74739.892262]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:27.869454+00:00 bl-cassoop-p11 kernel: [74739.892265]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:49:27.869456+00:00 bl-cassoop-p11 kernel: [74739.892269]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.869457+00:00 bl-cassoop-p11 kernel: [74739.892272]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.869459+00:00 bl-cassoop-p11 kernel: [74739.892276]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.869463+00:00 bl-cassoop-p11 kernel: [74739.892279]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:27.869465+00:00 bl-cassoop-p11 kernel: [74739.892282]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.869466+00:00 bl-cassoop-p11 kernel: [74739.892286]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:27.869468+00:00 bl-cassoop-p11 kernel: [74739.892289]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:27.869469+00:00 bl-cassoop-p11 kernel: [74739.892292]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:27.869471+00:00 bl-cassoop-p11 kernel: [74739.892296]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:27.869472+00:00 bl-cassoop-p11 kernel: [74739.892299]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:27.869476+00:00 bl-cassoop-p11 kernel: [74739.892302]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:27.869478+00:00 bl-cassoop-p11 kernel: [74739.892306]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.869479+00:00 bl-cassoop-p11 kernel: [74739.892309]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:27.869481+00:00 bl-cassoop-p11 kernel: [74739.892313]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:27.869484+00:00 bl-cassoop-p11 kernel: [74739.892314] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:27.869485+00:00 bl-cassoop-p11 kernel: [74739.892345] NMI backtrace for cpu 19
2014-01-18T13:49:27.869487+00:00 bl-cassoop-p11 kernel: [74739.892348] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.165 msecs
2014-01-18T13:49:27.869491+00:00 bl-cassoop-p11 kernel: [74739.892352] CPU: 19 PID: 0 Comm: swapper/19 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869493+00:00 bl-cassoop-p11 kernel: [74739.892354] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869494+00:00 bl-cassoop-p11 kernel: [74739.892362] task: ffff8817fadb8000 ti: ffff8817fadb6000 task.ti: ffff8817fadb6000
2014-01-18T13:49:27.869496+00:00 bl-cassoop-p11 kernel: [74739.892372] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869498+00:00 bl-cassoop-p11 kernel: [74739.892388] RSP: 0018:ffff8817fadb7da8  EFLAGS: 00000046
2014-01-18T13:49:27.869542+00:00 bl-cassoop-p11 kernel: [74739.892399] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869543+00:00 bl-cassoop-p11 kernel: [74739.892410] RDX: 0000000000000000 RSI: ffff8817fadb7fd8 RDI: 0000000000000013
2014-01-18T13:49:27.869548+00:00 bl-cassoop-p11 kernel: [74739.892420] RBP: ffff8817fadb7dd8 R08: 0000000000012051 R09: 0000000061ee44e4
2014-01-18T13:49:27.869549+00:00 bl-cassoop-p11 kernel: [74739.892430] R10: 000000000000183f R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.869551+00:00 bl-cassoop-p11 kernel: [74739.892441] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.869552+00:00 bl-cassoop-p11 kernel: [74739.892451] FS:  0000000000000000(0000) GS:ffff88301f320000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869554+00:00 bl-cassoop-p11 kernel: [74739.892461] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869555+00:00 bl-cassoop-p11 kernel: [74739.892462] CR2: 00000006dc018550 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869556+00:00 bl-cassoop-p11 kernel: [74739.892464] Stack:
2014-01-18T13:49:27.869558+00:00 bl-cassoop-p11 kernel: [74739.892465]  ffff8817fadb7dd8 00000013810bc824 ffffe8ffffb20320 000043f364f6ca30
2014-01-18T13:49:27.869562+00:00 bl-cassoop-p11 kernel: [74739.892470]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadb7e38 ffffffff815e5a9f
2014-01-18T13:49:27.869563+00:00 bl-cassoop-p11 kernel: [74739.892475]  0000000000000000 0000000009cb9094 0000000000000000 0000000009cb9094
2014-01-18T13:49:27.869564+00:00 bl-cassoop-p11 kernel: [74739.892479] Call Trace:
2014-01-18T13:49:27.869566+00:00 bl-cassoop-p11 kernel: [74739.892482]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.869568+00:00 bl-cassoop-p11 kernel: [74739.892485]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.869569+00:00 bl-cassoop-p11 kernel: [74739.892489]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.869571+00:00 bl-cassoop-p11 kernel: [74739.892492]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.869573+00:00 bl-cassoop-p11 kernel: [74739.892495]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.869576+00:00 bl-cassoop-p11 kernel: [74739.892499]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.869579+00:00 bl-cassoop-p11 kernel: [74739.892501] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.869580+00:00 bl-cassoop-p11 kernel: [74739.892532] NMI backtrace for cpu 20
2014-01-18T13:49:27.869582+00:00 bl-cassoop-p11 kernel: [74739.892535] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.252 msecs
2014-01-18T13:49:27.869584+00:00 bl-cassoop-p11 kernel: [74739.892540] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869585+00:00 bl-cassoop-p11 kernel: [74739.892542] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869589+00:00 bl-cassoop-p11 kernel: [74739.892544] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:49:27.869592+00:00 bl-cassoop-p11 kernel: [74739.892546] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:27.869593+00:00 bl-cassoop-p11 kernel: [74739.892550] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:49:27.869595+00:00 bl-cassoop-p11 kernel: [74739.892556] RAX: 0000000000000002 RBX: ffff88181fb55088 RCX: ffff88181fa387c0
2014-01-18T13:49:27.869596+00:00 bl-cassoop-p11 kernel: [74739.892560] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:27.869598+00:00 bl-cassoop-p11 kernel: [74739.892564] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:49:27.869599+00:00 bl-cassoop-p11 kernel: [74739.892570] R10: 0000000000000004 R11: 0000000000000206 R12: ffff88181fb55080
2014-01-18T13:49:27.869603+00:00 bl-cassoop-p11 kernel: [74739.892573] R13: ffffffff8105ae90 R14: ffff8801111b3d50 R15: 0000000000015040
2014-01-18T13:49:27.869604+00:00 bl-cassoop-p11 kernel: [74739.892578] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869606+00:00 bl-cassoop-p11 kernel: [74739.892581] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869607+00:00 bl-cassoop-p11 kernel: [74739.892585] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:27.869608+00:00 bl-cassoop-p11 kernel: [74739.892590] Stack:
2014-01-18T13:49:27.869610+00:00 bl-cassoop-p11 kernel: [74739.892594]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:49:27.869611+00:00 bl-cassoop-p11 kernel: [74739.892607]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:49:27.869613+00:00 bl-cassoop-p11 kernel: [74739.892612]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:49:27.869623+00:00 bl-cassoop-p11 kernel: [74739.892617] Call Trace:
2014-01-18T13:49:27.869624+00:00 bl-cassoop-p11 kernel: [74739.892622]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.869626+00:00 bl-cassoop-p11 kernel: [74739.892626]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.869628+00:00 bl-cassoop-p11 kernel: [74739.892630]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:27.869629+00:00 bl-cassoop-p11 kernel: [74739.892634]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:27.869631+00:00 bl-cassoop-p11 kernel: [74739.892637]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:27.869632+00:00 bl-cassoop-p11 kernel: [74739.892641]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:27.869634+00:00 bl-cassoop-p11 kernel: [74739.892645]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:27.869638+00:00 bl-cassoop-p11 kernel: [74739.892649]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:27.869639+00:00 bl-cassoop-p11 kernel: [74739.892653]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:27.869642+00:00 bl-cassoop-p11 kernel: [74739.892655] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:27.869643+00:00 bl-cassoop-p11 kernel: [74739.892688] NMI backtrace for cpu 21
2014-01-18T13:49:27.869688+00:00 bl-cassoop-p11 kernel: [74739.892691] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.308 msecs
2014-01-18T13:49:27.869690+00:00 bl-cassoop-p11 kernel: [74739.892697] CPU: 21 PID: 0 Comm: swapper/21 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869694+00:00 bl-cassoop-p11 kernel: [74739.892698] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869696+00:00 bl-cassoop-p11 kernel: [74739.892699] task: ffff8817fadbaee0 ti: ffff8817fadc2000 task.ti: ffff8817fadc2000
2014-01-18T13:49:27.869703+00:00 bl-cassoop-p11 kernel: [74739.892701] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869705+00:00 bl-cassoop-p11 kernel: [74739.892704] RSP: 0018:ffff8817fadc3da8  EFLAGS: 00000046
2014-01-18T13:49:27.869706+00:00 bl-cassoop-p11 kernel: [74739.892705] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869708+00:00 bl-cassoop-p11 kernel: [74739.892715] RDX: 0000000000000000 RSI: ffff8817fadc3fd8 RDI: 0000000000000015
2014-01-18T13:49:27.869709+00:00 bl-cassoop-p11 kernel: [74739.892725] RBP: ffff8817fadc3dd8 R08: 000000000001ad51 R09: 00000000000226ef
2014-01-18T13:49:27.869710+00:00 bl-cassoop-p11 kernel: [74739.892734] R10: 0000000000001697 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.869715+00:00 bl-cassoop-p11 kernel: [74739.892743] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.869717+00:00 bl-cassoop-p11 kernel: [74739.892752] FS:  0000000000000000(0000) GS:ffff88301f340000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869718+00:00 bl-cassoop-p11 kernel: [74739.892762] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869720+00:00 bl-cassoop-p11 kernel: [74739.892771] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869720+00:00 bl-cassoop-p11 kernel: [74739.892780] Stack:
2014-01-18T13:49:27.869722+00:00 bl-cassoop-p11 kernel: [74739.892788]  ffff8817fadc3dd8 00000015810bc824 ffffe8ffffb40320 000043f364f88641
2014-01-18T13:49:27.869723+00:00 bl-cassoop-p11 kernel: [74739.892801]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc3e38 ffffffff815e5a9f
2014-01-18T13:49:27.869727+00:00 bl-cassoop-p11 kernel: [74739.892803]  0000000000000000 0000000015e7be04 0000000000000000 0000000015e7be04
2014-01-18T13:49:27.869729+00:00 bl-cassoop-p11 kernel: [74739.892806] Call Trace:
2014-01-18T13:49:27.869730+00:00 bl-cassoop-p11 kernel: [74739.892809]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.869732+00:00 bl-cassoop-p11 kernel: [74739.892812]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.869733+00:00 bl-cassoop-p11 kernel: [74739.892814]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.869735+00:00 bl-cassoop-p11 kernel: [74739.892817]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.869737+00:00 bl-cassoop-p11 kernel: [74739.892819]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.869738+00:00 bl-cassoop-p11 kernel: [74739.892822]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.869744+00:00 bl-cassoop-p11 kernel: [74739.892823] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.869745+00:00 bl-cassoop-p11 kernel: [74739.892844] NMI backtrace for cpu 22
2014-01-18T13:49:27.869747+00:00 bl-cassoop-p11 kernel: [74739.892847] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.340 msecs
2014-01-18T13:49:27.869749+00:00 bl-cassoop-p11 kernel: [74739.892851] CPU: 22 PID: 0 Comm: swapper/22 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869750+00:00 bl-cassoop-p11 kernel: [74739.892853] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869752+00:00 bl-cassoop-p11 kernel: [74739.892855] task: ffff8817fadbc650 ti: ffff8817fadc4000 task.ti: ffff8817fadc4000
2014-01-18T13:49:27.869756+00:00 bl-cassoop-p11 kernel: [74739.892856] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869758+00:00 bl-cassoop-p11 kernel: [74739.892859] RSP: 0018:ffff8817fadc5da8  EFLAGS: 00000046
2014-01-18T13:49:27.869759+00:00 bl-cassoop-p11 kernel: [74739.892861] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869761+00:00 bl-cassoop-p11 kernel: [74739.892862] RDX: 0000000000000000 RSI: ffff8817fadc5fd8 RDI: 0000000000000016
2014-01-18T13:49:27.869762+00:00 bl-cassoop-p11 kernel: [74739.892864] RBP: ffff8817fadc5dd8 R08: 0000000000000c35 R09: 000000000041c310
2014-01-18T13:49:27.869764+00:00 bl-cassoop-p11 kernel: [74739.892865] R10: 000000000000195e R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.869765+00:00 bl-cassoop-p11 kernel: [74739.892867] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.869767+00:00 bl-cassoop-p11 kernel: [74739.892869] FS:  0000000000000000(0000) GS:ffff88181fb60000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869770+00:00 bl-cassoop-p11 kernel: [74739.892870] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869772+00:00 bl-cassoop-p11 kernel: [74739.892872] CR2: 000000008b06d0b0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869773+00:00 bl-cassoop-p11 kernel: [74739.892873] Stack:
2014-01-18T13:49:27.869775+00:00 bl-cassoop-p11 kernel: [74739.892874]  ffff8817fadc5dd8 00000016810bc824 ffffe8e800360320 000043f36553b996
2014-01-18T13:49:27.869776+00:00 bl-cassoop-p11 kernel: [74739.892879]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc5e38 ffffffff815e5a9f
2014-01-18T13:49:27.869778+00:00 bl-cassoop-p11 kernel: [74739.892884]  0000000000000000 00000000008db76c 0000000000000000 00000000008db76c
2014-01-18T13:49:27.869779+00:00 bl-cassoop-p11 kernel: [74739.892888] Call Trace:
2014-01-18T13:49:27.869783+00:00 bl-cassoop-p11 kernel: [74739.892892]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.869784+00:00 bl-cassoop-p11 kernel: [74739.892903]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.869786+00:00 bl-cassoop-p11 kernel: [74739.892914]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.869787+00:00 bl-cassoop-p11 kernel: [74739.892926]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.869789+00:00 bl-cassoop-p11 kernel: [74739.892937]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.869832+00:00 bl-cassoop-p11 kernel: [74739.892948]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.869835+00:00 bl-cassoop-p11 kernel: [74739.892954] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.869836+00:00 bl-cassoop-p11 kernel: [74739.892987] NMI backtrace for cpu 23
2014-01-18T13:49:27.869841+00:00 bl-cassoop-p11 kernel: [74739.892990] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.409 msecs
2014-01-18T13:49:27.869843+00:00 bl-cassoop-p11 kernel: [74739.892994] CPU: 23 PID: 0 Comm: swapper/23 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869844+00:00 bl-cassoop-p11 kernel: [74739.892995] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869846+00:00 bl-cassoop-p11 kernel: [74739.892997] task: ffff8817fadbddc0 ti: ffff8817fadc6000 task.ti: ffff8817fadc6000
2014-01-18T13:49:27.869848+00:00 bl-cassoop-p11 kernel: [74739.892998] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869849+00:00 bl-cassoop-p11 kernel: [74739.893001] RSP: 0018:ffff8817fadc7da8  EFLAGS: 00000046
2014-01-18T13:49:27.869851+00:00 bl-cassoop-p11 kernel: [74739.893002] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869855+00:00 bl-cassoop-p11 kernel: [74739.893003] RDX: 0000000000000000 RSI: ffff8817fadc7fd8 RDI: 0000000000000017
2014-01-18T13:49:27.869856+00:00 bl-cassoop-p11 kernel: [74739.893004] RBP: ffff8817fadc7dd8 R08: 0000000000016e39 R09: 00000001e8147119
2014-01-18T13:49:27.869858+00:00 bl-cassoop-p11 kernel: [74739.893005] R10: 0000000000001fa9 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.869859+00:00 bl-cassoop-p11 kernel: [74739.893006] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.869861+00:00 bl-cassoop-p11 kernel: [74739.893008] FS:  0000000000000000(0000) GS:ffff88301f360000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869862+00:00 bl-cassoop-p11 kernel: [74739.893009] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869864+00:00 bl-cassoop-p11 kernel: [74739.893010] CR2: 00007fe8cd9a3000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869867+00:00 bl-cassoop-p11 kernel: [74739.893011] Stack:
2014-01-18T13:49:27.869869+00:00 bl-cassoop-p11 kernel: [74739.893012]  ffff8817fadc7dd8 00000017810bc824 ffffe8ffffb60320 000043f364fa35c2
2014-01-18T13:49:27.869870+00:00 bl-cassoop-p11 kernel: [74739.893015]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadc7e38 ffffffff815e5a9f
2014-01-18T13:49:27.869872+00:00 bl-cassoop-p11 kernel: [74739.893018]  0000000000000000 00000000118aab15 0000000000000000 00000000118aab15
2014-01-18T13:49:27.869873+00:00 bl-cassoop-p11 kernel: [74739.893021] Call Trace:
2014-01-18T13:49:27.869874+00:00 bl-cassoop-p11 kernel: [74739.893023]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.869876+00:00 bl-cassoop-p11 kernel: [74739.893025]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.869878+00:00 bl-cassoop-p11 kernel: [74739.893028]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.869882+00:00 bl-cassoop-p11 kernel: [74739.893030]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.869883+00:00 bl-cassoop-p11 kernel: [74739.893033]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.869885+00:00 bl-cassoop-p11 kernel: [74739.893035]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.869888+00:00 bl-cassoop-p11 kernel: [74739.893036] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.869889+00:00 bl-cassoop-p11 kernel: [74739.893159] NMI backtrace for cpu 24
2014-01-18T13:49:27.869891+00:00 bl-cassoop-p11 kernel: [74739.893162] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.482 msecs
2014-01-18T13:49:27.869892+00:00 bl-cassoop-p11 kernel: [74739.893166] CPU: 24 PID: 0 Comm: swapper/24 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869896+00:00 bl-cassoop-p11 kernel: [74739.893168] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869898+00:00 bl-cassoop-p11 kernel: [74739.893169] task: ffff8817fadc8000 ti: ffff8817fadd0000 task.ti: ffff8817fadd0000
2014-01-18T13:49:27.869900+00:00 bl-cassoop-p11 kernel: [74739.893171] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869901+00:00 bl-cassoop-p11 kernel: [74739.893174] RSP: 0018:ffff8817fadd1da8  EFLAGS: 00000046
2014-01-18T13:49:27.869903+00:00 bl-cassoop-p11 kernel: [74739.893175] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869904+00:00 bl-cassoop-p11 kernel: [74739.893177] RDX: 0000000000000000 RSI: ffff8817fadd1fd8 RDI: 0000000000000018
2014-01-18T13:49:27.869906+00:00 bl-cassoop-p11 kernel: [74739.893179] RBP: ffff8817fadd1dd8 R08: 00000000000166d1 R09: 0000000073d5ea90
2014-01-18T13:49:27.869911+00:00 bl-cassoop-p11 kernel: [74739.893180] R10: 0000000000001c5b R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.869912+00:00 bl-cassoop-p11 kernel: [74739.893182] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.869914+00:00 bl-cassoop-p11 kernel: [74739.893184] FS:  0000000000000000(0000) GS:ffff88181fb80000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.869915+00:00 bl-cassoop-p11 kernel: [74739.893185] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.869917+00:00 bl-cassoop-p11 kernel: [74739.893187] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.869918+00:00 bl-cassoop-p11 kernel: [74739.893188] Stack:
2014-01-18T13:49:27.869919+00:00 bl-cassoop-p11 kernel: [74739.893189]  ffff8817fadd1dd8 00000018810bc824 ffffe8e800380320 000043f364f9711e
2014-01-18T13:49:27.869923+00:00 bl-cassoop-p11 kernel: [74739.893194]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd1e38 ffffffff815e5a9f
2014-01-18T13:49:27.869925+00:00 bl-cassoop-p11 kernel: [74739.893199]  0000000000000000 000000001c9ca8e4 0000000000000000 000000001c9ca8e4
2014-01-18T13:49:27.869926+00:00 bl-cassoop-p11 kernel: [74739.893203] Call Trace:
2014-01-18T13:49:27.869969+00:00 bl-cassoop-p11 kernel: [74739.893206]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.869971+00:00 bl-cassoop-p11 kernel: [74739.893209]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.869972+00:00 bl-cassoop-p11 kernel: [74739.893213]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.869974+00:00 bl-cassoop-p11 kernel: [74739.893216]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.869975+00:00 bl-cassoop-p11 kernel: [74739.893219]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.869979+00:00 bl-cassoop-p11 kernel: [74739.893223]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.869982+00:00 bl-cassoop-p11 kernel: [74739.893224] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.869983+00:00 bl-cassoop-p11 kernel: [74739.893256] NMI backtrace for cpu 25
2014-01-18T13:49:27.869985+00:00 bl-cassoop-p11 kernel: [74739.893260] CPU: 25 PID: 0 Comm: swapper/25 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.869987+00:00 bl-cassoop-p11 kernel: [74739.893261] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.869988+00:00 bl-cassoop-p11 kernel: [74739.893263] task: ffff8817fadc9770 ti: ffff8817fadd2000 task.ti: ffff8817fadd2000
2014-01-18T13:49:27.869990+00:00 bl-cassoop-p11 kernel: [74739.893264] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.869994+00:00 bl-cassoop-p11 kernel: [74739.893267] RSP: 0018:ffff8817fadd3da8  EFLAGS: 00000046
2014-01-18T13:49:27.869996+00:00 bl-cassoop-p11 kernel: [74739.893268] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.869997+00:00 bl-cassoop-p11 kernel: [74739.893270] RDX: 0000000000000000 RSI: ffff8817fadd3fd8 RDI: 0000000000000019
2014-01-18T13:49:27.869999+00:00 bl-cassoop-p11 kernel: [74739.893271] RBP: ffff8817fadd3dd8 R08: 0000000000013afd R09: 00000000000348ea
2014-01-18T13:49:27.870000+00:00 bl-cassoop-p11 kernel: [74739.893272] R10: 0000000000001fe2 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.870002+00:00 bl-cassoop-p11 kernel: [74739.893278] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.870003+00:00 bl-cassoop-p11 kernel: [74739.893288] FS:  0000000000000000(0000) GS:ffff88301f380000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870008+00:00 bl-cassoop-p11 kernel: [74739.893297] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870009+00:00 bl-cassoop-p11 kernel: [74739.893307] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870010+00:00 bl-cassoop-p11 kernel: [74739.893315] Stack:
2014-01-18T13:49:27.870012+00:00 bl-cassoop-p11 kernel: [74739.893324]  ffff8817fadd3dd8 00000019810bc824 ffffe8ffffb80320 000043f364fb0db5
2014-01-18T13:49:27.870013+00:00 bl-cassoop-p11 kernel: [74739.893363]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd3e38 ffffffff815e5a9f
2014-01-18T13:49:27.870015+00:00 bl-cassoop-p11 kernel: [74739.893371]  0000000000000000 000000001b667901 0000000000000000 000000001b667901
2014-01-18T13:49:27.870016+00:00 bl-cassoop-p11 kernel: [74739.893374] Call Trace:
2014-01-18T13:49:27.870018+00:00 bl-cassoop-p11 kernel: [74739.893377]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.870022+00:00 bl-cassoop-p11 kernel: [74739.893379]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.870023+00:00 bl-cassoop-p11 kernel: [74739.893381]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.870025+00:00 bl-cassoop-p11 kernel: [74739.893384]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.870027+00:00 bl-cassoop-p11 kernel: [74739.893387]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.870028+00:00 bl-cassoop-p11 kernel: [74739.893389]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.870031+00:00 bl-cassoop-p11 kernel: [74739.893390] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.870032+00:00 bl-cassoop-p11 kernel: [74739.893411] NMI backtrace for cpu 27
2014-01-18T13:49:27.870036+00:00 bl-cassoop-p11 kernel: [74739.893412] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.514 msecs
2014-01-18T13:49:27.870038+00:00 bl-cassoop-p11 kernel: [74739.893415] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870040+00:00 bl-cassoop-p11 kernel: [74739.893416] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870041+00:00 bl-cassoop-p11 kernel: [74739.893417] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:49:27.870044+00:00 bl-cassoop-p11 kernel: [74739.893418] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:27.870045+00:00 bl-cassoop-p11 kernel: [74739.893420] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:49:27.870046+00:00 bl-cassoop-p11 kernel: [74739.893421] RAX: 0000000000000030 RBX: ffff881887d73470 RCX: 0000000000000010
2014-01-18T13:49:27.870050+00:00 bl-cassoop-p11 kernel: [74739.893422] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:49:27.870052+00:00 bl-cassoop-p11 kernel: [74739.893423] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:49:27.870053+00:00 bl-cassoop-p11 kernel: [74739.893424] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 000000000000000f
2014-01-18T13:49:27.870055+00:00 bl-cassoop-p11 kernel: [74739.893425] R13: ffff88301f2f5040 R14: ffff88301f2f5050 R15: 0000000000000001
2014-01-18T13:49:27.870056+00:00 bl-cassoop-p11 kernel: [74739.893426] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870058+00:00 bl-cassoop-p11 kernel: [74739.893427] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870059+00:00 bl-cassoop-p11 kernel: [74739.893428] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:27.870060+00:00 bl-cassoop-p11 kernel: [74739.893429] Stack:
2014-01-18T13:49:27.870064+00:00 bl-cassoop-p11 kernel: [74739.893430]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:49:27.870111+00:00 bl-cassoop-p11 kernel: [74739.893433]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:49:27.870113+00:00 bl-cassoop-p11 kernel: [74739.893435]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:49:27.870114+00:00 bl-cassoop-p11 kernel: [74739.893438] Call Trace:
2014-01-18T13:49:27.870115+00:00 bl-cassoop-p11 kernel: [74739.893452]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870117+00:00 bl-cassoop-p11 kernel: [74739.893454]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.870119+00:00 bl-cassoop-p11 kernel: [74739.893456]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870123+00:00 bl-cassoop-p11 kernel: [74739.893459]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870124+00:00 bl-cassoop-p11 kernel: [74739.893461]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.870126+00:00 bl-cassoop-p11 kernel: [74739.893463]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.870128+00:00 bl-cassoop-p11 kernel: [74739.893466]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:27.870129+00:00 bl-cassoop-p11 kernel: [74739.893468]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:27.870131+00:00 bl-cassoop-p11 kernel: [74739.893485]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:27.870133+00:00 bl-cassoop-p11 kernel: [74739.893502]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:49:27.870134+00:00 bl-cassoop-p11 kernel: [74739.893519]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:27.870139+00:00 bl-cassoop-p11 kernel: [74739.893536]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:27.870140+00:00 bl-cassoop-p11 kernel: [74739.893552]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:27.870142+00:00 bl-cassoop-p11 kernel: [74739.893569]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:27.870144+00:00 bl-cassoop-p11 kernel: [74739.893575]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:27.870145+00:00 bl-cassoop-p11 kernel: [74739.893577]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:27.870147+00:00 bl-cassoop-p11 kernel: [74739.893580]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:27.870148+00:00 bl-cassoop-p11 kernel: [74739.893582]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:27.870152+00:00 bl-cassoop-p11 kernel: [74739.893584]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:27.870154+00:00 bl-cassoop-p11 kernel: [74739.893587]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:27.870155+00:00 bl-cassoop-p11 kernel: [74739.893589]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:27.870157+00:00 bl-cassoop-p11 kernel: [74739.893592]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:27.870158+00:00 bl-cassoop-p11 kernel: [74739.893594]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:49:27.870160+00:00 bl-cassoop-p11 kernel: [74739.893596]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:27.870162+00:00 bl-cassoop-p11 kernel: [74739.893599]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:27.870163+00:00 bl-cassoop-p11 kernel: [74739.893601]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:27.870168+00:00 bl-cassoop-p11 kernel: [74739.893604]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:27.870169+00:00 bl-cassoop-p11 kernel: [74739.893606]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:27.870171+00:00 bl-cassoop-p11 kernel: [74739.893609]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:27.870173+00:00 bl-cassoop-p11 kernel: [74739.893611]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:27.870174+00:00 bl-cassoop-p11 kernel: [74739.893613]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:27.870176+00:00 bl-cassoop-p11 kernel: [74739.893616]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:27.870177+00:00 bl-cassoop-p11 kernel: [74739.893617]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:27.870181+00:00 bl-cassoop-p11 kernel: [74739.893620]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:27.870183+00:00 bl-cassoop-p11 kernel: [74739.893622]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:27.870184+00:00 bl-cassoop-p11 kernel: [74739.893624]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:27.870186+00:00 bl-cassoop-p11 kernel: [74739.893626]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:27.870188+00:00 bl-cassoop-p11 kernel: [74739.893628]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:27.870189+00:00 bl-cassoop-p11 kernel: [74739.893630]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:27.870190+00:00 bl-cassoop-p11 kernel: [74739.893632]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:27.870192+00:00 bl-cassoop-p11 kernel: [74739.893635]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:27.870199+00:00 bl-cassoop-p11 kernel: [74739.893636] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:27.870201+00:00 bl-cassoop-p11 kernel: [74739.893656] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.681 msecs
2014-01-18T13:49:27.870202+00:00 bl-cassoop-p11 kernel: [74739.893658] NMI backtrace for cpu 26
2014-01-18T13:49:27.870204+00:00 bl-cassoop-p11 kernel: [74739.893661] CPU: 26 PID: 0 Comm: swapper/26 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870205+00:00 bl-cassoop-p11 kernel: [74739.893663] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870207+00:00 bl-cassoop-p11 kernel: [74739.893666] task: ffff8817fadcaee0 ti: ffff8817fadd4000 task.ti: ffff8817fadd4000
2014-01-18T13:49:27.870211+00:00 bl-cassoop-p11 kernel: [74739.893671] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.870213+00:00 bl-cassoop-p11 kernel: [74739.893675] RSP: 0018:ffff8817fadd5da8  EFLAGS: 00000046
2014-01-18T13:49:27.870214+00:00 bl-cassoop-p11 kernel: [74739.893678] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.870216+00:00 bl-cassoop-p11 kernel: [74739.893682] RDX: 0000000000000000 RSI: ffff8817fadd5fd8 RDI: 000000000000001a
2014-01-18T13:49:27.870217+00:00 bl-cassoop-p11 kernel: [74739.893686] RBP: ffff8817fadd5dd8 R08: 0000000000019b71 R09: 00000000a74d9890
2014-01-18T13:49:27.870261+00:00 bl-cassoop-p11 kernel: [74739.893689] R10: 0000000000001c4f R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.870262+00:00 bl-cassoop-p11 kernel: [74739.893693] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.870264+00:00 bl-cassoop-p11 kernel: [74739.893697] FS:  0000000000000000(0000) GS:ffff88181fba0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870268+00:00 bl-cassoop-p11 kernel: [74739.893700] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870269+00:00 bl-cassoop-p11 kernel: [74739.893704] CR2: 000000008b06c008 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870271+00:00 bl-cassoop-p11 kernel: [74739.893707] Stack:
2014-01-18T13:49:27.870272+00:00 bl-cassoop-p11 kernel: [74739.893709]  ffff8817fadd5dd8 0000001a810bc824 ffffe8e8003a0320 000043f364fda67f
2014-01-18T13:49:27.870274+00:00 bl-cassoop-p11 kernel: [74739.893730]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd5e38 ffffffff815e5a9f
2014-01-18T13:49:27.870275+00:00 bl-cassoop-p11 kernel: [74739.893752]  0000000000000000 00000000160dbd12 0000000000000000 00000000160dbd12
2014-01-18T13:49:27.870276+00:00 bl-cassoop-p11 kernel: [74739.893759] Call Trace:
2014-01-18T13:49:27.870284+00:00 bl-cassoop-p11 kernel: [74739.893762]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.870286+00:00 bl-cassoop-p11 kernel: [74739.893765]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.870287+00:00 bl-cassoop-p11 kernel: [74739.893769]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.870289+00:00 bl-cassoop-p11 kernel: [74739.893772]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.870291+00:00 bl-cassoop-p11 kernel: [74739.893775]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.870292+00:00 bl-cassoop-p11 kernel: [74739.893779]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.870295+00:00 bl-cassoop-p11 kernel: [74739.893780] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.870296+00:00 bl-cassoop-p11 kernel: [74739.893810] NMI backtrace for cpu 28
2014-01-18T13:49:27.870301+00:00 bl-cassoop-p11 kernel: [74739.893812] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.835 msecs
2014-01-18T13:49:27.870302+00:00 bl-cassoop-p11 kernel: [74739.893815] CPU: 28 PID: 0 Comm: swapper/28 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870304+00:00 bl-cassoop-p11 kernel: [74739.893817] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870306+00:00 bl-cassoop-p11 kernel: [74739.893819] task: ffff8817fadcddc0 ti: ffff8817fadd8000 task.ti: ffff8817fadd8000
2014-01-18T13:49:27.870308+00:00 bl-cassoop-p11 kernel: [74739.893820] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.870309+00:00 bl-cassoop-p11 kernel: [74739.893823] RSP: 0018:ffff8817fadd9da8  EFLAGS: 00000046
2014-01-18T13:49:27.870310+00:00 bl-cassoop-p11 kernel: [74739.893825] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.870314+00:00 bl-cassoop-p11 kernel: [74739.893826] RDX: 0000000000000000 RSI: ffff8817fadd9fd8 RDI: 0000000001c0d000
2014-01-18T13:49:27.870316+00:00 bl-cassoop-p11 kernel: [74739.893828] RBP: ffff8817fadd9dd8 R08: 00000000000017d1 R09: 000000000000a256
2014-01-18T13:49:27.870318+00:00 bl-cassoop-p11 kernel: [74739.893830] R10: 0000000000001bd2 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.870319+00:00 bl-cassoop-p11 kernel: [74739.893831] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.870321+00:00 bl-cassoop-p11 kernel: [74739.893833] FS:  0000000000000000(0000) GS:ffff88181fbc0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870322+00:00 bl-cassoop-p11 kernel: [74739.893835] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870323+00:00 bl-cassoop-p11 kernel: [74739.893836] CR2: 000000008b06d0f0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870327+00:00 bl-cassoop-p11 kernel: [74739.893838] Stack:
2014-01-18T13:49:27.870329+00:00 bl-cassoop-p11 kernel: [74739.893839]  ffff8817fadd9dd8 0000001c810bc824 ffffe8e8003c0320 000043f365765e41
2014-01-18T13:49:27.870330+00:00 bl-cassoop-p11 kernel: [74739.893844]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817fadd9e38 ffffffff815e5a9f
2014-01-18T13:49:27.870332+00:00 bl-cassoop-p11 kernel: [74739.893862]  0000000000000000 0000000002fb6543 0000000000000000 0000000002fb6543
2014-01-18T13:49:27.870333+00:00 bl-cassoop-p11 kernel: [74739.893884] Call Trace:
2014-01-18T13:49:27.870334+00:00 bl-cassoop-p11 kernel: [74739.893895]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.870336+00:00 bl-cassoop-p11 kernel: [74739.893904]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.870337+00:00 bl-cassoop-p11 kernel: [74739.893908]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.870342+00:00 bl-cassoop-p11 kernel: [74739.893911]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.870344+00:00 bl-cassoop-p11 kernel: [74739.893915]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.870345+00:00 bl-cassoop-p11 kernel: [74739.893918]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.870348+00:00 bl-cassoop-p11 kernel: [74739.893920] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.870349+00:00 bl-cassoop-p11 kernel: [74739.893951] NMI backtrace for cpu 29
2014-01-18T13:49:27.870351+00:00 bl-cassoop-p11 kernel: [74739.893954] CPU: 29 PID: 0 Comm: swapper/29 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870352+00:00 bl-cassoop-p11 kernel: [74739.893955] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870357+00:00 bl-cassoop-p11 kernel: [74739.893957] task: ffff8817fade0000 ti: ffff8817fadda000 task.ti: ffff8817fadda000
2014-01-18T13:49:27.870359+00:00 bl-cassoop-p11 kernel: [74739.893959] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.870360+00:00 bl-cassoop-p11 kernel: [74739.893962] RSP: 0018:ffff8817faddbda8  EFLAGS: 00000046
2014-01-18T13:49:27.870405+00:00 bl-cassoop-p11 kernel: [74739.893963] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.870407+00:00 bl-cassoop-p11 kernel: [74739.893965] RDX: 0000000000000000 RSI: ffff8817faddbfd8 RDI: 000000000000001d
2014-01-18T13:49:27.870408+00:00 bl-cassoop-p11 kernel: [74739.893966] RBP: ffff8817faddbdd8 R08: 000000000004cc71 R09: 000000024409ce40
2014-01-18T13:49:27.870410+00:00 bl-cassoop-p11 kernel: [74739.893968] R10: 0000000000001822 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.870415+00:00 bl-cassoop-p11 kernel: [74739.893969] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.870417+00:00 bl-cassoop-p11 kernel: [74739.893971] FS:  0000000000000000(0000) GS:ffff88301f3c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870419+00:00 bl-cassoop-p11 kernel: [74739.893973] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870420+00:00 bl-cassoop-p11 kernel: [74739.893975] CR2: 00007f0b82eaf1a8 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870421+00:00 bl-cassoop-p11 kernel: [74739.893976] Stack:
2014-01-18T13:49:27.870422+00:00 bl-cassoop-p11 kernel: [74739.893977]  ffff8817faddbdd8 0000001d810bc824 ffffe8ffffbc0320 000043f365022e09
2014-01-18T13:49:27.870424+00:00 bl-cassoop-p11 kernel: [74739.893982]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817faddbe38 ffffffff815e5a9f
2014-01-18T13:49:27.870425+00:00 bl-cassoop-p11 kernel: [74739.893986]  0000000000000000 00000000054bcb82 0000000000000000 00000000054bcb82
2014-01-18T13:49:27.870429+00:00 bl-cassoop-p11 kernel: [74739.893991] Call Trace:
2014-01-18T13:49:27.870431+00:00 bl-cassoop-p11 kernel: [74739.893994]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.870432+00:00 bl-cassoop-p11 kernel: [74739.893997]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.870434+00:00 bl-cassoop-p11 kernel: [74739.894001]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.870435+00:00 bl-cassoop-p11 kernel: [74739.894015]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.870437+00:00 bl-cassoop-p11 kernel: [74739.894033]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.870438+00:00 bl-cassoop-p11 kernel: [74739.894050]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.870444+00:00 bl-cassoop-p11 kernel: [74739.894060] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:27.870445+00:00 bl-cassoop-p11 kernel: [74739.894129] NMI backtrace for cpu 30
2014-01-18T13:49:27.870447+00:00 bl-cassoop-p11 kernel: [74739.894132] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870449+00:00 bl-cassoop-p11 kernel: [74739.894134] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870450+00:00 bl-cassoop-p11 kernel: [74739.894135] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:49:27.870452+00:00 bl-cassoop-p11 kernel: [74739.894137] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:27.870453+00:00 bl-cassoop-p11 kernel: [74739.894140] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:49:27.870457+00:00 bl-cassoop-p11 kernel: [74739.894141] RAX: 0000000000000286 RBX: ffff88188471f8d0 RCX: 0000000000000001
2014-01-18T13:49:27.870459+00:00 bl-cassoop-p11 kernel: [74739.894143] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:27.870460+00:00 bl-cassoop-p11 kernel: [74739.894145] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:49:27.870462+00:00 bl-cassoop-p11 kernel: [74739.894146] R10: 00000000000035de R11: 0000000000000001 R12: 000000000000000f
2014-01-18T13:49:27.870463+00:00 bl-cassoop-p11 kernel: [74739.894148] R13: ffff88301f2f5040 R14: ffff88301f2f5050 R15: 0000000000000001
2014-01-18T13:49:27.870465+00:00 bl-cassoop-p11 kernel: [74739.894150] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870466+00:00 bl-cassoop-p11 kernel: [74739.894152] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870468+00:00 bl-cassoop-p11 kernel: [74739.894153] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870471+00:00 bl-cassoop-p11 kernel: [74739.894154] Stack:
2014-01-18T13:49:27.870473+00:00 bl-cassoop-p11 kernel: [74739.894155]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:49:27.870474+00:00 bl-cassoop-p11 kernel: [74739.894160]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:49:27.870475+00:00 bl-cassoop-p11 kernel: [74739.894164]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:49:27.870476+00:00 bl-cassoop-p11 kernel: [74739.894169] Call Trace:
2014-01-18T13:49:27.870478+00:00 bl-cassoop-p11 kernel: [74739.894172]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870480+00:00 bl-cassoop-p11 kernel: [74739.894175]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:27.870485+00:00 bl-cassoop-p11 kernel: [74739.894179]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870486+00:00 bl-cassoop-p11 kernel: [74739.894182]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:27.870488+00:00 bl-cassoop-p11 kernel: [74739.894185]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:27.870490+00:00 bl-cassoop-p11 kernel: [74739.894188]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:49:27.870491+00:00 bl-cassoop-p11 kernel: [74739.894191]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:27.870493+00:00 bl-cassoop-p11 kernel: [74739.894195]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:27.870495+00:00 bl-cassoop-p11 kernel: [74739.894207]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:49:27.870496+00:00 bl-cassoop-p11 kernel: [74739.894217]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:49:27.870500+00:00 bl-cassoop-p11 kernel: [74739.894228]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:49:27.870502+00:00 bl-cassoop-p11 kernel: [74739.894239]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:49:27.870504+00:00 bl-cassoop-p11 kernel: [74739.894249]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:49:27.870505+00:00 bl-cassoop-p11 kernel: [74739.894259]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:49:27.870540+00:00 bl-cassoop-p11 kernel: [74739.894263]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:49:27.870542+00:00 bl-cassoop-p11 kernel: [74739.894266]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:49:27.870544+00:00 bl-cassoop-p11 kernel: [74739.894269]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:49:27.870548+00:00 bl-cassoop-p11 kernel: [74739.894272]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:49:27.870549+00:00 bl-cassoop-p11 kernel: [74739.894275]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:49:27.870551+00:00 bl-cassoop-p11 kernel: [74739.894279]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:49:27.870553+00:00 bl-cassoop-p11 kernel: [74739.894282]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:49:27.870554+00:00 bl-cassoop-p11 kernel: [74739.894285]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:27.870556+00:00 bl-cassoop-p11 kernel: [74739.894289]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:27.870557+00:00 bl-cassoop-p11 kernel: [74739.894292]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:27.870559+00:00 bl-cassoop-p11 kernel: [74739.894295]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:27.870563+00:00 bl-cassoop-p11 kernel: [74739.894299]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:49:27.870564+00:00 bl-cassoop-p11 kernel: [74739.894302]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:27.870566+00:00 bl-cassoop-p11 kernel: [74739.894305]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:49:27.870569+00:00 bl-cassoop-p11 kernel: [74739.894307] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:27.870570+00:00 bl-cassoop-p11 kernel: [74739.894338] NMI backtrace for cpu 31
2014-01-18T13:49:27.870572+00:00 bl-cassoop-p11 kernel: [74739.894341] CPU: 31 PID: 0 Comm: swapper/31 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:27.870573+00:00 bl-cassoop-p11 kernel: [74739.894343] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:27.870577+00:00 bl-cassoop-p11 kernel: [74739.894344] task: ffff8817fade2ee0 ti: ffff8817fadde000 task.ti: ffff8817fadde000
2014-01-18T13:49:27.870579+00:00 bl-cassoop-p11 kernel: [74739.894346] RIP: 0010:[<ffffffff813ef41e>]  [<ffffffff813ef41e>] intel_idle+0xae/0x110
2014-01-18T13:49:27.870580+00:00 bl-cassoop-p11 kernel: [74739.894349] RSP: 0018:ffff8817faddfda8  EFLAGS: 00000046
2014-01-18T13:49:27.870582+00:00 bl-cassoop-p11 kernel: [74739.894350] RAX: 0000000000000030 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:27.870583+00:00 bl-cassoop-p11 kernel: [74739.894352] RDX: 0000000000000000 RSI: ffff8817faddffd8 RDI: 000000000000001f
2014-01-18T13:49:27.870585+00:00 bl-cassoop-p11 kernel: [74739.894353] RBP: ffff8817faddfdd8 R08: 0000000000012825 R09: 00000000353f7981
2014-01-18T13:49:27.870586+00:00 bl-cassoop-p11 kernel: [74739.894355] R10: 00000000000019d2 R11: 0000000000000000 R12: 0000000000000005
2014-01-18T13:49:27.870590+00:00 bl-cassoop-p11 kernel: [74739.894366] R13: 0000000000000030 R14: 0000000000000004 R15: 0000000000000005
2014-01-18T13:49:27.870592+00:00 bl-cassoop-p11 kernel: [74739.894375] FS:  0000000000000000(0000) GS:ffff88301f3e0000(0000) knlGS:0000000000000000
2014-01-18T13:49:27.870593+00:00 bl-cassoop-p11 kernel: [74739.894386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:27.870595+00:00 bl-cassoop-p11 kernel: [74739.894396] CR2: 0000000001ed5cc0 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:27.870595+00:00 bl-cassoop-p11 kernel: [74739.894405] Stack:
2014-01-18T13:49:27.870597+00:00 bl-cassoop-p11 kernel: [74739.894414]  ffff8817faddfdd8 0000001f810bc824 ffffe8ffffbe0320 000043f365056cc5
2014-01-18T13:49:27.870598+00:00 bl-cassoop-p11 kernel: [74739.894453]  ffffffff81c8fa30 ffffffff81c8f860 ffff8817faddfe38 ffffffff815e5a9f
2014-01-18T13:49:27.870600+00:00 bl-cassoop-p11 kernel: [74739.894461]  0000000000000000 00000000083a60b7 0000000000000000 00000000083a60b7
2014-01-18T13:49:27.870603+00:00 bl-cassoop-p11 kernel: [74739.894466] Call Trace:
2014-01-18T13:49:27.870605+00:00 bl-cassoop-p11 kernel: [74739.894469]  [<ffffffff815e5a9f>] cpuidle_enter_state+0x4f/0xe0
2014-01-18T13:49:27.870607+00:00 bl-cassoop-p11 kernel: [74739.894472]  [<ffffffff815e5bf0>] cpuidle_idle_call+0xc0/0x220
2014-01-18T13:49:27.870608+00:00 bl-cassoop-p11 kernel: [74739.894475]  [<ffffffff8101d0ee>] arch_cpu_idle+0xe/0x30
2014-01-18T13:49:27.870610+00:00 bl-cassoop-p11 kernel: [74739.894478]  [<ffffffff810ba768>] cpu_idle_loop+0x98/0x260
2014-01-18T13:49:27.870611+00:00 bl-cassoop-p11 kernel: [74739.894481]  [<ffffffff810ba99b>] cpu_startup_entry+0x6b/0x70
2014-01-18T13:49:27.870613+00:00 bl-cassoop-p11 kernel: [74739.894485]  [<ffffffff81042038>] start_secondary+0xc8/0xd0
2014-01-18T13:49:27.870618+00:00 bl-cassoop-p11 kernel: [74739.894486] Code: 38 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <85> 1d d4 07 8a 00 75 0e 48 8d 75 dc bf 05 00 00 00 e8 ec 3a cd
2014-01-18T13:49:55.444150+00:00 bl-cassoop-p11 kernel: [74751.532920] ------------[ cut here ]------------
2014-01-18T13:49:55.446145+00:00 bl-cassoop-p11 kernel: [74751.532926] WARNING: CPU: 3 PID: 5299 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:49:55.446188+00:00 bl-cassoop-p11 kernel: [74751.532928] Watchdog detected hard LOCKUP on cpu 3
2014-01-18T13:49:55.446220+00:00 bl-cassoop-p11 kernel: [74751.532929] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.446257+00:00 bl-cassoop-p11 kernel: [74751.532945] CPU: 3 PID: 5299 Comm: nrpe Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.446287+00:00 bl-cassoop-p11 kernel: [74751.532946] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.446317+00:00 bl-cassoop-p11 kernel: [74751.532947]  00000000000000f5 ffff88301f227ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:49:55.446347+00:00 bl-cassoop-p11 kernel: [74751.532952]  ffff88301f227bf8 ffff88301f227be8 ffffffff810653ac 00000000000123f3
2014-01-18T13:49:55.446375+00:00 bl-cassoop-p11 kernel: [74751.532955]  ffff882ffa430000 0000000000000000 ffff88301f227d20 0000000000000000
2014-01-18T13:49:55.446403+00:00 bl-cassoop-p11 kernel: [74751.532957] Call Trace:
2014-01-18T13:49:55.446432+00:00 bl-cassoop-p11 kernel: [74751.532958]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:49:55.446462+00:00 bl-cassoop-p11 kernel: [74751.532964]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:49:55.446491+00:00 bl-cassoop-p11 kernel: [74751.532967]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:49:55.446520+00:00 bl-cassoop-p11 kernel: [74751.532969]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:49:55.446549+00:00 bl-cassoop-p11 kernel: [74751.532972]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:49:55.446578+00:00 bl-cassoop-p11 kernel: [74751.532976]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:49:55.446606+00:00 bl-cassoop-p11 kernel: [74751.532978]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:49:55.446635+00:00 bl-cassoop-p11 kernel: [74751.532981]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:49:55.446664+00:00 bl-cassoop-p11 kernel: [74751.532984]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:49:55.446693+00:00 bl-cassoop-p11 kernel: [74751.532987]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:49:55.446722+00:00 bl-cassoop-p11 kernel: [74751.532990]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:49:55.446761+00:00 bl-cassoop-p11 kernel: [74751.532993]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:49:55.446790+00:00 bl-cassoop-p11 kernel: [74751.532995]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:49:55.446819+00:00 bl-cassoop-p11 kernel: [74751.532998]  [<ffffffff81748ea8>] default_do_nmi+0x58/0x240
2014-01-18T13:49:55.446848+00:00 bl-cassoop-p11 kernel: [74751.533000]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:49:55.446879+00:00 bl-cassoop-p11 kernel: [74751.533003]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:49:55.446908+00:00 bl-cassoop-p11 kernel: [74751.533005]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.446937+00:00 bl-cassoop-p11 kernel: [74751.533008]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.446977+00:00 bl-cassoop-p11 kernel: [74751.533010]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.447007+00:00 bl-cassoop-p11 kernel: [74751.533011]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:55.447036+00:00 bl-cassoop-p11 kernel: [74751.533015]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:49:55.447065+00:00 bl-cassoop-p11 kernel: [74751.533017]  [<ffffffff811d1165>] ? fd_install+0x25/0x30
2014-01-18T13:49:55.447093+00:00 bl-cassoop-p11 kernel: [74751.533019]  [<ffffffff8162162a>] ? SYSC_accept4+0x17a/0x240
2014-01-18T13:49:55.447122+00:00 bl-cassoop-p11 kernel: [74751.533021]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:49:55.447150+00:00 bl-cassoop-p11 kernel: [74751.533023]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:49:55.447179+00:00 bl-cassoop-p11 kernel: [74751.533025]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:49:55.447207+00:00 bl-cassoop-p11 kernel: [74751.533028]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:49:55.447235+00:00 bl-cassoop-p11 kernel: [74751.533030]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:55.447263+00:00 bl-cassoop-p11 kernel: [74751.533032] ---[ end trace 51e5206791572f01 ]---
2014-01-18T13:49:55.447292+00:00 bl-cassoop-p11 kernel: [74765.534409] ------------[ cut here ]------------
2014-01-18T13:49:55.447321+00:00 bl-cassoop-p11 kernel: [74765.534421] WARNING: CPU: 2 PID: 42018 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:49:55.447350+00:00 bl-cassoop-p11 kernel: [74765.534423] Watchdog detected hard LOCKUP on cpu 2
2014-01-18T13:49:55.447381+00:00 bl-cassoop-p11 kernel: [74765.534424] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.447410+00:00 bl-cassoop-p11 kernel: [74765.534446] CPU: 2 PID: 42018 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.447439+00:00 bl-cassoop-p11 kernel: [74765.534448] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.447468+00:00 bl-cassoop-p11 kernel: [74765.534449]  00000000000000f5 ffff88181fa27ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:49:55.447497+00:00 bl-cassoop-p11 kernel: [74765.534456]  ffff88181fa27bf8 ffff88181fa27be8 ffffffff810653ac 00000000000123f3
2014-01-18T13:49:55.447570+00:00 bl-cassoop-p11 kernel: [74765.534461]  ffff8817fafe0000 0000000000000000 ffff88181fa27d20 0000000000000000
2014-01-18T13:49:55.447600+00:00 bl-cassoop-p11 kernel: [74765.534465] Call Trace:
2014-01-18T13:49:55.447629+00:00 bl-cassoop-p11 kernel: [74765.534466]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:49:55.447658+00:00 bl-cassoop-p11 kernel: [74765.534473]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:49:55.447694+00:00 bl-cassoop-p11 kernel: [74765.534476]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:49:55.447724+00:00 bl-cassoop-p11 kernel: [74765.534480]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:49:55.447753+00:00 bl-cassoop-p11 kernel: [74765.534483]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:49:55.447782+00:00 bl-cassoop-p11 kernel: [74765.534487]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:49:55.447811+00:00 bl-cassoop-p11 kernel: [74765.534490]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:49:55.447839+00:00 bl-cassoop-p11 kernel: [74765.534493]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:49:55.447868+00:00 bl-cassoop-p11 kernel: [74765.534497]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:49:55.447905+00:00 bl-cassoop-p11 kernel: [74765.534500]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:49:55.447933+00:00 bl-cassoop-p11 kernel: [74765.534504]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:49:55.447962+00:00 bl-cassoop-p11 kernel: [74765.534508]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:49:55.447991+00:00 bl-cassoop-p11 kernel: [74765.534511]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:49:55.448019+00:00 bl-cassoop-p11 kernel: [74765.534515]  [<ffffffff81748f39>] default_do_nmi+0xe9/0x240
2014-01-18T13:49:55.448048+00:00 bl-cassoop-p11 kernel: [74765.534518]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:49:55.448076+00:00 bl-cassoop-p11 kernel: [74765.534522]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:49:55.448104+00:00 bl-cassoop-p11 kernel: [74765.534526]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.448133+00:00 bl-cassoop-p11 kernel: [74765.534529]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.448161+00:00 bl-cassoop-p11 kernel: [74765.534532]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:49:55.448189+00:00 bl-cassoop-p11 kernel: [74765.534533]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:49:55.448218+00:00 bl-cassoop-p11 kernel: [74765.534539]  [<ffffffff810676e5>] forget_original_parent+0x35/0x1a0
2014-01-18T13:49:55.448246+00:00 bl-cassoop-p11 kernel: [74765.534543]  [<ffffffff81067867>] exit_notify+0x17/0x110
2014-01-18T13:49:55.448275+00:00 bl-cassoop-p11 kernel: [74765.534546]  [<ffffffff81067f84>] do_exit+0x1f4/0x480
2014-01-18T13:49:55.448303+00:00 bl-cassoop-p11 kernel: [74765.534549]  [<ffffffff81068257>] SyS_exit+0x17/0x20
2014-01-18T13:49:55.448418+00:00 bl-cassoop-p11 kernel: [74765.534552]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:55.448447+00:00 bl-cassoop-p11 kernel: [74765.534554] ---[ end trace 51e5206791572f02 ]---
2014-01-18T13:49:55.448475+00:00 bl-cassoop-p11 kernel: [74767.474155] BUG: soft lockup - CPU#1 stuck for 22s! [collectd:5830]
2014-01-18T13:49:55.448599+00:00 bl-cassoop-p11 kernel: [74767.474158] BUG: soft lockup - CPU#0 stuck for 22s! [java:42506]
2014-01-18T13:49:55.448710+00:00 bl-cassoop-p11 kernel: [74767.474175] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.448743+00:00 bl-cassoop-p11 kernel: [74767.474177] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.448772+00:00 bl-cassoop-p11 kernel: [74767.474178] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.448800+00:00 bl-cassoop-p11 kernel: [74767.474179] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:49:55.448829+00:00 bl-cassoop-p11 kernel: [74767.474182] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:55.448865+00:00 bl-cassoop-p11 kernel: [74767.474183] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:49:55.448894+00:00 bl-cassoop-p11 kernel: [74767.474184] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa97868
2014-01-18T13:49:55.448922+00:00 bl-cassoop-p11 kernel: [74767.474185] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.448950+00:00 bl-cassoop-p11 kernel: [74767.474185] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:49:55.448978+00:00 bl-cassoop-p11 kernel: [74767.474186] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:49:55.449006+00:00 bl-cassoop-p11 kernel: [74767.474187] R13: ffff880045ccfc48 R14: 0000000000000292 R15: ffff880045ccfc78
2014-01-18T13:49:55.449041+00:00 bl-cassoop-p11 kernel: [74767.474189] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.449070+00:00 bl-cassoop-p11 kernel: [74767.474190] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.449101+00:00 bl-cassoop-p11 kernel: [74767.474191] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:49:55.449129+00:00 bl-cassoop-p11 kernel: [74767.474191] Stack:
2014-01-18T13:49:55.449157+00:00 bl-cassoop-p11 kernel: [74767.474196]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:49:55.449189+00:00 bl-cassoop-p11 kernel: [74767.474199]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:49:55.449217+00:00 bl-cassoop-p11 kernel: [74767.474203]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:49:55.449245+00:00 bl-cassoop-p11 kernel: [74767.474203] Call Trace:
2014-01-18T13:49:55.449274+00:00 bl-cassoop-p11 kernel: [74767.474207]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.449302+00:00 bl-cassoop-p11 kernel: [74767.474210]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:55.449331+00:00 bl-cassoop-p11 kernel: [74767.474213]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:55.449394+00:00 bl-cassoop-p11 kernel: [74767.474215]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:55.449424+00:00 bl-cassoop-p11 kernel: [74767.474217]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:55.449453+00:00 bl-cassoop-p11 kernel: [74767.474220]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:55.449482+00:00 bl-cassoop-p11 kernel: [74767.474222]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:55.449510+00:00 bl-cassoop-p11 kernel: [74767.474225]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:55.449539+00:00 bl-cassoop-p11 kernel: [74767.474228]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:55.449568+00:00 bl-cassoop-p11 kernel: [74767.474247] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:55.449599+00:00 bl-cassoop-p11 kernel: [74767.480394] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.449629+00:00 bl-cassoop-p11 kernel: [74767.480418] CPU: 1 PID: 5830 Comm: collectd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.449657+00:00 bl-cassoop-p11 kernel: [74767.480420] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.449686+00:00 bl-cassoop-p11 kernel: [74767.480422] task: ffff882ff0912ee0 ti: ffff882ffa6f6000 task.ti: ffff882ffa6f6000
2014-01-18T13:49:55.449733+00:00 bl-cassoop-p11 kernel: [74767.480424] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:55.449762+00:00 bl-cassoop-p11 kernel: [74767.480430] RSP: 0018:ffff882ffa6f7d08  EFLAGS: 00000202
2014-01-18T13:49:55.449792+00:00 bl-cassoop-p11 kernel: [74767.480431] RAX: 0000000000000286 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:49:55.449820+00:00 bl-cassoop-p11 kernel: [74767.480433] RDX: ffff88181fa38680 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:55.449849+00:00 bl-cassoop-p11 kernel: [74767.480434] RBP: ffff882ffa6f7d48 R08: 0000000000000000 R09: 000000000000730e
2014-01-18T13:49:55.449877+00:00 bl-cassoop-p11 kernel: [74767.480436] R10: 0000000000003708 R11: 0000000000000293 R12: 0000000000000000
2014-01-18T13:49:55.449905+00:00 bl-cassoop-p11 kernel: [74767.480437] R13: 00007f2a9217e000 R14: ffff88307fff9d80 R15: ffff882ffa6f7f24
2014-01-18T13:49:55.449940+00:00 bl-cassoop-p11 kernel: [74767.480440] FS:  00007f2a8a89f700(0000) GS:ffff88301f200000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.449968+00:00 bl-cassoop-p11 kernel: [74767.480442] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.449997+00:00 bl-cassoop-p11 kernel: [74767.480443] CR2: 00007f2a92182000 CR3: 0000002ff0b0e000 CR4: 00000000000407e0
2014-01-18T13:49:55.450024+00:00 bl-cassoop-p11 kernel: [74767.480445] Stack:
2014-01-18T13:49:55.450052+00:00 bl-cassoop-p11 kernel: [74767.480446]  ffff882ff7817000 ffff88181fa386a8 00000007f2a9217e 0000000000000002
2014-01-18T13:49:55.450080+00:00 bl-cassoop-p11 kernel: [74767.480453]  ffffffff8139a810 0000000000000001 ffffffff81d03da0 00007f2a92182000
2014-01-18T13:49:55.450108+00:00 bl-cassoop-p11 kernel: [74767.480457]  ffff882ffa6f7db8 ffffffff810cb5a5 ffff882ff0912ee0 0000000000000001
2014-01-18T13:49:55.450136+00:00 bl-cassoop-p11 kernel: [74767.480462] Call Trace:
2014-01-18T13:49:55.450167+00:00 bl-cassoop-p11 kernel: [74767.480468]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:55.450195+00:00 bl-cassoop-p11 kernel: [74767.480471]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:55.450223+00:00 bl-cassoop-p11 kernel: [74767.480474]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:49:55.450252+00:00 bl-cassoop-p11 kernel: [74767.480478]  [<ffffffff8139ab4e>] rdmsr_on_cpu+0x4e/0x70
2014-01-18T13:49:55.450280+00:00 bl-cassoop-p11 kernel: [74767.480484]  [<ffffffffa001b62b>] show_temp+0xab/0xf0 [coretemp]
2014-01-18T13:49:55.450308+00:00 bl-cassoop-p11 kernel: [74767.480488]  [<ffffffff81490f20>] dev_attr_show+0x20/0x60
2014-01-18T13:49:55.450337+00:00 bl-cassoop-p11 kernel: [74767.480492]  [<ffffffff8114d37e>] ? __get_free_pages+0xe/0x40
2014-01-18T13:49:55.450365+00:00 bl-cassoop-p11 kernel: [74767.480497]  [<ffffffff81227e86>] fill_read_buffer.isra.9+0x66/0xf0
2014-01-18T13:49:55.450400+00:00 bl-cassoop-p11 kernel: [74767.480500]  [<ffffffff81227fc0>] sysfs_read_file+0xb0/0xf0
2014-01-18T13:49:55.450429+00:00 bl-cassoop-p11 kernel: [74767.480505]  [<ffffffff811b42e4>] vfs_read+0xb4/0x180
2014-01-18T13:49:55.450457+00:00 bl-cassoop-p11 kernel: [74767.480509]  [<ffffffff811b4562>] SyS_read+0x52/0xa0
2014-01-18T13:49:55.450485+00:00 bl-cassoop-p11 kernel: [74767.480513]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:55.450514+00:00 bl-cassoop-p11 kernel: [74767.480515] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:55.552807+00:00 bl-cassoop-p11 kernel: [74767.586195] BUG: soft lockup - CPU#10 stuck for 22s! [java:42435]
2014-01-18T13:49:55.552936+00:00 bl-cassoop-p11 kernel: [74767.589120] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.552978+00:00 bl-cassoop-p11 kernel: [74767.589144] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.553008+00:00 bl-cassoop-p11 kernel: [74767.589145] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.553040+00:00 bl-cassoop-p11 kernel: [74767.589147] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:49:55.553070+00:00 bl-cassoop-p11 kernel: [74767.589148] RIP: 0010:[<ffffffff81731076>]  [<ffffffff81731076>] oom_kill_process.part.6+0xbc/0x2cf
2014-01-18T13:49:55.553098+00:00 bl-cassoop-p11 kernel: [74767.589151] RSP: 0000:ffff8819e9a81a28  EFLAGS: 00000286
2014-01-18T13:49:55.553126+00:00 bl-cassoop-p11 kernel: [74767.589153] RAX: ffff882ff0e31b00 RBX: 00000000000000d0 RCX: 0000000000000006
2014-01-18T13:49:55.553161+00:00 bl-cassoop-p11 kernel: [74767.589154] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffffffff81c06040
2014-01-18T13:49:55.553190+00:00 bl-cassoop-p11 kernel: [74767.589155] RBP: ffff8819e9a81a88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:49:55.553218+00:00 bl-cassoop-p11 kernel: [74767.589156] R10: 0000000000000e32 R11: 0000000000005000 R12: ffff882ff044e3f0
2014-01-18T13:49:55.553247+00:00 bl-cassoop-p11 kernel: [74767.589157] R13: 0000000000000005 R14: 0000000000000246 R15: 0000000028e74000
2014-01-18T13:49:55.553276+00:00 bl-cassoop-p11 kernel: [74767.589158] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.553304+00:00 bl-cassoop-p11 kernel: [74767.589160] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.553332+00:00 bl-cassoop-p11 kernel: [74767.589161] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:55.553360+00:00 bl-cassoop-p11 kernel: [74767.589162] Stack:
2014-01-18T13:49:55.553388+00:00 bl-cassoop-p11 kernel: [74767.589163]  ffff8819e9a81a38 ffff882ff044ddc0 000000000114eccd ffffc90028e74000
2014-01-18T13:49:55.553416+00:00 bl-cassoop-p11 kernel: [74767.589166]  ffff8819e9a81a88 ffff882ff0e31b00 ffff882ff5c55dc0 000000000114eccd
2014-01-18T13:49:55.553444+00:00 bl-cassoop-p11 kernel: [74767.589169]  0000000000000000 ffffc90028e74000 ffff882ff0b86a00 ffff882ff044ddc0
2014-01-18T13:49:55.553473+00:00 bl-cassoop-p11 kernel: [74767.589172] Call Trace:
2014-01-18T13:49:55.553502+00:00 bl-cassoop-p11 kernel: [74767.589175]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:49:55.553531+00:00 bl-cassoop-p11 kernel: [74767.589178]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:49:55.553560+00:00 bl-cassoop-p11 kernel: [74767.589180]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:49:55.553588+00:00 bl-cassoop-p11 kernel: [74767.589182]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:49:55.553616+00:00 bl-cassoop-p11 kernel: [74767.589185]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:49:55.553647+00:00 bl-cassoop-p11 kernel: [74767.589188]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:49:55.553676+00:00 bl-cassoop-p11 kernel: [74767.589190]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:55.553733+00:00 bl-cassoop-p11 kernel: [74767.589193]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:55.553767+00:00 bl-cassoop-p11 kernel: [74767.589195]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:55.553796+00:00 bl-cassoop-p11 kernel: [74767.589198]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:55.553825+00:00 bl-cassoop-p11 kernel: [74767.589200]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:55.553854+00:00 bl-cassoop-p11 kernel: [74767.589203]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:55.553882+00:00 bl-cassoop-p11 kernel: [74767.589206]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:55.553918+00:00 bl-cassoop-p11 kernel: [74767.589208]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:55.553948+00:00 bl-cassoop-p11 kernel: [74767.589210]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:49:55.553977+00:00 bl-cassoop-p11 kernel: [74767.589212]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:55.554005+00:00 bl-cassoop-p11 kernel: [74767.589215]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:55.554033+00:00 bl-cassoop-p11 kernel: [74767.589218]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:55.554062+00:00 bl-cassoop-p11 kernel: [74767.589220]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:55.554090+00:00 bl-cassoop-p11 kernel: [74767.589222]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:55.554125+00:00 bl-cassoop-p11 kernel: [74767.589224]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:55.554155+00:00 bl-cassoop-p11 kernel: [74767.589227]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:55.554183+00:00 bl-cassoop-p11 kernel: [74767.589229]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:55.554212+00:00 bl-cassoop-p11 kernel: [74767.589230] Code: 48 c7 c7 40 60 c0 81 4d 89 fe 45 31 ed e8 63 65 01 00 4c 89 7d a8 49 8b 9e 00 03 00 00 4d 8d a6 00 03 00 00 48 81 eb 10 03 00 00 <eb> 53 49 8b 87 a0 02 00 00 48 39 83 a0 02 00 00 74 35 48 8b 4d
2014-01-18T13:49:55.576647+00:00 bl-cassoop-p11 kernel: [74767.610202] BUG: soft lockup - CPU#12 stuck for 22s! [jsvc:7242]
2014-01-18T13:49:55.576772+00:00 bl-cassoop-p11 kernel: [74767.612997] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.576806+00:00 bl-cassoop-p11 kernel: [74767.613013] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.576836+00:00 bl-cassoop-p11 kernel: [74767.613014] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.576865+00:00 bl-cassoop-p11 kernel: [74767.613015] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:49:55.576894+00:00 bl-cassoop-p11 kernel: [74767.613017] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:55.576923+00:00 bl-cassoop-p11 kernel: [74767.613020] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:49:55.576951+00:00 bl-cassoop-p11 kernel: [74767.613021] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa38680
2014-01-18T13:49:55.576984+00:00 bl-cassoop-p11 kernel: [74767.613022] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.577013+00:00 bl-cassoop-p11 kernel: [74767.613023] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:49:55.577041+00:00 bl-cassoop-p11 kernel: [74767.613024] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff8817f37ca000
2014-01-18T13:49:55.577069+00:00 bl-cassoop-p11 kernel: [74767.613025] R13: 0000000000000001 R14: 0000000000000292 R15: ffff8817f8e45938
2014-01-18T13:49:55.577098+00:00 bl-cassoop-p11 kernel: [74767.613027] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.577126+00:00 bl-cassoop-p11 kernel: [74767.613028] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.577154+00:00 bl-cassoop-p11 kernel: [74767.613029] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:55.577189+00:00 bl-cassoop-p11 kernel: [74767.613030] Stack:
2014-01-18T13:49:55.577217+00:00 bl-cassoop-p11 kernel: [74767.613031]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:49:55.577245+00:00 bl-cassoop-p11 kernel: [74767.613035]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:49:55.577283+00:00 bl-cassoop-p11 kernel: [74767.613037]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:49:55.577311+00:00 bl-cassoop-p11 kernel: [74767.613040] Call Trace:
2014-01-18T13:49:55.577341+00:00 bl-cassoop-p11 kernel: [74767.613044]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.577369+00:00 bl-cassoop-p11 kernel: [74767.613047]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.577398+00:00 bl-cassoop-p11 kernel: [74767.613049]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:55.577427+00:00 bl-cassoop-p11 kernel: [74767.613051]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:55.577456+00:00 bl-cassoop-p11 kernel: [74767.613054]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:55.577492+00:00 bl-cassoop-p11 kernel: [74767.613056]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:55.577521+00:00 bl-cassoop-p11 kernel: [74767.613058]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:55.577550+00:00 bl-cassoop-p11 kernel: [74767.613060]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:55.577578+00:00 bl-cassoop-p11 kernel: [74767.613062]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:55.577607+00:00 bl-cassoop-p11 kernel: [74767.613064]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:55.577636+00:00 bl-cassoop-p11 kernel: [74767.613066]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:55.577664+00:00 bl-cassoop-p11 kernel: [74767.613069]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:55.577693+00:00 bl-cassoop-p11 kernel: [74767.613071]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:55.577731+00:00 bl-cassoop-p11 kernel: [74767.613073]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:49:55.577760+00:00 bl-cassoop-p11 kernel: [74767.613075]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:55.577789+00:00 bl-cassoop-p11 kernel: [74767.613078]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:55.577817+00:00 bl-cassoop-p11 kernel: [74767.613080]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:55.577846+00:00 bl-cassoop-p11 kernel: [74767.613082]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:55.577875+00:00 bl-cassoop-p11 kernel: [74767.613083] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:55.588633+00:00 bl-cassoop-p11 kernel: [74767.622200] BUG: soft lockup - CPU#13 stuck for 22s! [kswapd1:284]
2014-01-18T13:49:55.588757+00:00 bl-cassoop-p11 kernel: [74767.624962] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.588791+00:00 bl-cassoop-p11 kernel: [74767.624978] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.588820+00:00 bl-cassoop-p11 kernel: [74767.624980] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.588848+00:00 bl-cassoop-p11 kernel: [74767.624981] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:49:55.588878+00:00 bl-cassoop-p11 kernel: [74767.624982] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:55.588907+00:00 bl-cassoop-p11 kernel: [74767.624985] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:49:55.588934+00:00 bl-cassoop-p11 kernel: [74767.624986] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa386a8
2014-01-18T13:49:55.588972+00:00 bl-cassoop-p11 kernel: [74767.624987] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.589009+00:00 bl-cassoop-p11 kernel: [74767.624988] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:49:55.589038+00:00 bl-cassoop-p11 kernel: [74767.624989] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000009
2014-01-18T13:49:55.589066+00:00 bl-cassoop-p11 kernel: [74767.624991] R13: ffffffff8105ae90 R14: 0000000000000292 R15: ffff882ff8363748
2014-01-18T13:49:55.589094+00:00 bl-cassoop-p11 kernel: [74767.624992] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.589122+00:00 bl-cassoop-p11 kernel: [74767.624993] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.589161+00:00 bl-cassoop-p11 kernel: [74767.624994] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:55.589190+00:00 bl-cassoop-p11 kernel: [74767.624995] Stack:
2014-01-18T13:49:55.589218+00:00 bl-cassoop-p11 kernel: [74767.624996]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:49:55.589246+00:00 bl-cassoop-p11 kernel: [74767.625000]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:49:55.589274+00:00 bl-cassoop-p11 kernel: [74767.625003]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:49:55.589302+00:00 bl-cassoop-p11 kernel: [74767.625005] Call Trace:
2014-01-18T13:49:55.589331+00:00 bl-cassoop-p11 kernel: [74767.625011]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.589359+00:00 bl-cassoop-p11 kernel: [74767.625014]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.589388+00:00 bl-cassoop-p11 kernel: [74767.625017]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:55.589416+00:00 bl-cassoop-p11 kernel: [74767.625019]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:55.589445+00:00 bl-cassoop-p11 kernel: [74767.625022]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:55.589473+00:00 bl-cassoop-p11 kernel: [74767.625024]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:55.589501+00:00 bl-cassoop-p11 kernel: [74767.625027]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:55.589530+00:00 bl-cassoop-p11 kernel: [74767.625030]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:55.589558+00:00 bl-cassoop-p11 kernel: [74767.625032]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:55.589587+00:00 bl-cassoop-p11 kernel: [74767.625035]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:55.589615+00:00 bl-cassoop-p11 kernel: [74767.625038]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:55.589644+00:00 bl-cassoop-p11 kernel: [74767.625040]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:55.589672+00:00 bl-cassoop-p11 kernel: [74767.625043]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:55.589708+00:00 bl-cassoop-p11 kernel: [74767.625045]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:55.589740+00:00 bl-cassoop-p11 kernel: [74767.625048]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:55.589771+00:00 bl-cassoop-p11 kernel: [74767.625051]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:55.589800+00:00 bl-cassoop-p11 kernel: [74767.625054]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:55.589828+00:00 bl-cassoop-p11 kernel: [74767.625056]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:55.589856+00:00 bl-cassoop-p11 kernel: [74767.625060]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:55.589885+00:00 bl-cassoop-p11 kernel: [74767.625062]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.589914+00:00 bl-cassoop-p11 kernel: [74767.625065]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:55.589950+00:00 bl-cassoop-p11 kernel: [74767.625067]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.589981+00:00 bl-cassoop-p11 kernel: [74767.625068] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:55.600714+00:00 bl-cassoop-p11 kernel: [74767.634214] BUG: soft lockup - CPU#14 stuck for 22s! [java:42064]
2014-01-18T13:49:55.600837+00:00 bl-cassoop-p11 kernel: [74767.636907] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.600878+00:00 bl-cassoop-p11 kernel: [74767.636930] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.600909+00:00 bl-cassoop-p11 kernel: [74767.636932] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.600937+00:00 bl-cassoop-p11 kernel: [74767.636934] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:49:55.600967+00:00 bl-cassoop-p11 kernel: [74767.636935] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:55.600996+00:00 bl-cassoop-p11 kernel: [74767.636939] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:49:55.601025+00:00 bl-cassoop-p11 kernel: [74767.636941] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa986d0
2014-01-18T13:49:55.601052+00:00 bl-cassoop-p11 kernel: [74767.636943] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.601081+00:00 bl-cassoop-p11 kernel: [74767.636945] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:49:55.601110+00:00 bl-cassoop-p11 kernel: [74767.636946] R10: 0000000000000004 R11: 0000000000000058 R12: ffffffff81049a3e
2014-01-18T13:49:55.601137+00:00 bl-cassoop-p11 kernel: [74767.636948] R13: ffff88195e365078 R14: 0000000000000292 R15: ffff88195e3650a8
2014-01-18T13:49:55.601165+00:00 bl-cassoop-p11 kernel: [74767.636950] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.601193+00:00 bl-cassoop-p11 kernel: [74767.636952] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.601222+00:00 bl-cassoop-p11 kernel: [74767.636953] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:49:55.601249+00:00 bl-cassoop-p11 kernel: [74767.636955] Stack:
2014-01-18T13:49:55.601278+00:00 bl-cassoop-p11 kernel: [74767.636956]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:49:55.601306+00:00 bl-cassoop-p11 kernel: [74767.636961]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:49:55.601334+00:00 bl-cassoop-p11 kernel: [74767.636966]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:49:55.601361+00:00 bl-cassoop-p11 kernel: [74767.636970] Call Trace:
2014-01-18T13:49:55.601390+00:00 bl-cassoop-p11 kernel: [74767.636974]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.601419+00:00 bl-cassoop-p11 kernel: [74767.636978]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.601447+00:00 bl-cassoop-p11 kernel: [74767.636981]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:55.601476+00:00 bl-cassoop-p11 kernel: [74767.636985]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:55.601505+00:00 bl-cassoop-p11 kernel: [74767.636988]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:55.601533+00:00 bl-cassoop-p11 kernel: [74767.636992]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:55.601562+00:00 bl-cassoop-p11 kernel: [74767.636995]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:55.601597+00:00 bl-cassoop-p11 kernel: [74767.636999]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:55.601627+00:00 bl-cassoop-p11 kernel: [74767.637002]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:55.601655+00:00 bl-cassoop-p11 kernel: [74767.637006]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:55.601684+00:00 bl-cassoop-p11 kernel: [74767.637009]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:55.601722+00:00 bl-cassoop-p11 kernel: [74767.637013]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:55.601752+00:00 bl-cassoop-p11 kernel: [74767.637016]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:55.601781+00:00 bl-cassoop-p11 kernel: [74767.637020]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:55.601816+00:00 bl-cassoop-p11 kernel: [74767.637023]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:55.601845+00:00 bl-cassoop-p11 kernel: [74767.637027]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:55.601874+00:00 bl-cassoop-p11 kernel: [74767.637031]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:55.601903+00:00 bl-cassoop-p11 kernel: [74767.637034]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:55.601931+00:00 bl-cassoop-p11 kernel: [74767.637038]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:55.601959+00:00 bl-cassoop-p11 kernel: [74767.637041]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:55.601988+00:00 bl-cassoop-p11 kernel: [74767.637045]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:49:55.602016+00:00 bl-cassoop-p11 kernel: [74767.637048]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:49:55.602044+00:00 bl-cassoop-p11 kernel: [74767.637052]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:49:55.602073+00:00 bl-cassoop-p11 kernel: [74767.637055]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:49:55.602101+00:00 bl-cassoop-p11 kernel: [74767.637059]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:55.602129+00:00 bl-cassoop-p11 kernel: [74767.637062]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:49:55.602158+00:00 bl-cassoop-p11 kernel: [74767.637066]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:49:55.602186+00:00 bl-cassoop-p11 kernel: [74767.637069]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:49:55.602214+00:00 bl-cassoop-p11 kernel: [74767.637072]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:49:55.602242+00:00 bl-cassoop-p11 kernel: [74767.637075]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:49:55.602271+00:00 bl-cassoop-p11 kernel: [74767.637079]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:49:55.602300+00:00 bl-cassoop-p11 kernel: [74767.637082]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:49:55.602344+00:00 bl-cassoop-p11 kernel: [74767.637085]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:49:55.602374+00:00 bl-cassoop-p11 kernel: [74767.637089]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:55.602403+00:00 bl-cassoop-p11 kernel: [74767.637092]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:49:55.602431+00:00 bl-cassoop-p11 kernel: [74767.637096]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:55.602460+00:00 bl-cassoop-p11 kernel: [74767.637099]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:49:55.602489+00:00 bl-cassoop-p11 kernel: [74767.637102]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:49:55.602517+00:00 bl-cassoop-p11 kernel: [74767.637105]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:49:55.602546+00:00 bl-cassoop-p11 kernel: [74767.637108]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:49:55.602580+00:00 bl-cassoop-p11 kernel: [74767.637111]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:55.602610+00:00 bl-cassoop-p11 kernel: [74767.637115]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:55.602641+00:00 bl-cassoop-p11 kernel: [74767.637118]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:55.602669+00:00 bl-cassoop-p11 kernel: [74767.637121]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:49:55.602697+00:00 bl-cassoop-p11 kernel: [74767.637124]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:49:55.602725+00:00 bl-cassoop-p11 kernel: [74767.637128]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:49:55.602754+00:00 bl-cassoop-p11 kernel: [74767.637131]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:55.602788+00:00 bl-cassoop-p11 kernel: [74767.637135]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:55.602818+00:00 bl-cassoop-p11 kernel: [74767.637136] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:55.624726+00:00 bl-cassoop-p11 kernel: [74767.658222] BUG: soft lockup - CPU#16 stuck for 22s! [kworker/u97:9:42017]
2014-01-18T13:49:55.624850+00:00 bl-cassoop-p11 kernel: [74767.660976] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.624885+00:00 bl-cassoop-p11 kernel: [74767.661010] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.624914+00:00 bl-cassoop-p11 kernel: [74767.661011] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.624943+00:00 bl-cassoop-p11 kernel: [74767.661015] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:49:55.624972+00:00 bl-cassoop-p11 kernel: [74767.661018] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:49:55.625001+00:00 bl-cassoop-p11 kernel: [74767.661020] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:55.625029+00:00 bl-cassoop-p11 kernel: [74767.661023] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:49:55.625057+00:00 bl-cassoop-p11 kernel: [74767.661025] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa98720
2014-01-18T13:49:55.625086+00:00 bl-cassoop-p11 kernel: [74767.661026] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.625114+00:00 bl-cassoop-p11 kernel: [74767.661028] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:49:55.625143+00:00 bl-cassoop-p11 kernel: [74767.661030] R10: 0000000000000004 R11: 0000000000000210 R12: ffffffff81254cb7
2014-01-18T13:49:55.625171+00:00 bl-cassoop-p11 kernel: [74767.661031] R13: ffff880075d0b5c8 R14: 0000000000000292 R15: ffff880075d0b5d8
2014-01-18T13:49:55.625200+00:00 bl-cassoop-p11 kernel: [74767.661033] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.625228+00:00 bl-cassoop-p11 kernel: [74767.661035] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.625256+00:00 bl-cassoop-p11 kernel: [74767.661037] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:55.625284+00:00 bl-cassoop-p11 kernel: [74767.661038] Stack:
2014-01-18T13:49:55.625311+00:00 bl-cassoop-p11 kernel: [74767.661040]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:49:55.625339+00:00 bl-cassoop-p11 kernel: [74767.661045]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:49:55.625376+00:00 bl-cassoop-p11 kernel: [74767.661049]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:49:55.625404+00:00 bl-cassoop-p11 kernel: [74767.661054] Call Trace:
2014-01-18T13:49:55.625432+00:00 bl-cassoop-p11 kernel: [74767.661058]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.625461+00:00 bl-cassoop-p11 kernel: [74767.661062]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.625490+00:00 bl-cassoop-p11 kernel: [74767.661065]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:55.625519+00:00 bl-cassoop-p11 kernel: [74767.661068]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:49:55.625547+00:00 bl-cassoop-p11 kernel: [74767.661071]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:49:55.625582+00:00 bl-cassoop-p11 kernel: [74767.661075]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:49:55.625611+00:00 bl-cassoop-p11 kernel: [74767.661078]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:49:55.625640+00:00 bl-cassoop-p11 kernel: [74767.661081]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:49:55.625669+00:00 bl-cassoop-p11 kernel: [74767.661085]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:49:55.625723+00:00 bl-cassoop-p11 kernel: [74767.661088]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:49:55.625759+00:00 bl-cassoop-p11 kernel: [74767.661092]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:49:55.625788+00:00 bl-cassoop-p11 kernel: [74767.661096]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:49:55.625817+00:00 bl-cassoop-p11 kernel: [74767.661099]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:49:55.625845+00:00 bl-cassoop-p11 kernel: [74767.661103]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:49:55.625874+00:00 bl-cassoop-p11 kernel: [74767.661106]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:49:55.625902+00:00 bl-cassoop-p11 kernel: [74767.661110]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:49:55.625930+00:00 bl-cassoop-p11 kernel: [74767.661113]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:49:55.625961+00:00 bl-cassoop-p11 kernel: [74767.661117]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:49:55.625989+00:00 bl-cassoop-p11 kernel: [74767.661120]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:49:55.626018+00:00 bl-cassoop-p11 kernel: [74767.661123]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:49:55.626046+00:00 bl-cassoop-p11 kernel: [74767.661127]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:49:55.626075+00:00 bl-cassoop-p11 kernel: [74767.661130]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:49:55.626103+00:00 bl-cassoop-p11 kernel: [74767.661133]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:49:55.626132+00:00 bl-cassoop-p11 kernel: [74767.661137]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:49:55.626160+00:00 bl-cassoop-p11 kernel: [74767.661140]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:49:55.626189+00:00 bl-cassoop-p11 kernel: [74767.661143]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:49:55.626217+00:00 bl-cassoop-p11 kernel: [74767.661147]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:55.626252+00:00 bl-cassoop-p11 kernel: [74767.661151]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.626281+00:00 bl-cassoop-p11 kernel: [74767.661154]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:55.626309+00:00 bl-cassoop-p11 kernel: [74767.661158]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.626340+00:00 bl-cassoop-p11 kernel: [74767.661159] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:55.636677+00:00 bl-cassoop-p11 kernel: [74767.670225] BUG: soft lockup - CPU#17 stuck for 22s! [java:42521]
2014-01-18T13:49:55.636801+00:00 bl-cassoop-p11 kernel: [74767.672976] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.636835+00:00 bl-cassoop-p11 kernel: [74767.672999] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.636865+00:00 bl-cassoop-p11 kernel: [74767.673001] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.636904+00:00 bl-cassoop-p11 kernel: [74767.673003] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:49:55.636934+00:00 bl-cassoop-p11 kernel: [74767.673004] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:49:55.636962+00:00 bl-cassoop-p11 kernel: [74767.673008] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:49:55.636991+00:00 bl-cassoop-p11 kernel: [74767.673009] RAX: 0000000000000006 RBX: ffffffff81d03da0 RCX: ffff88181fa78748
2014-01-18T13:49:55.637019+00:00 bl-cassoop-p11 kernel: [74767.673011] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.637047+00:00 bl-cassoop-p11 kernel: [74767.673013] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:49:55.637075+00:00 bl-cassoop-p11 kernel: [74767.673014] R10: 0000000000000004 R11: 0000000000017960 R12: ffff880cff722548
2014-01-18T13:49:55.637104+00:00 bl-cassoop-p11 kernel: [74767.673016] R13: 0000000000000000 R14: 0000000000000292 R15: ffff882feccbf938
2014-01-18T13:49:55.637132+00:00 bl-cassoop-p11 kernel: [74767.673018] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.637160+00:00 bl-cassoop-p11 kernel: [74767.673019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.637188+00:00 bl-cassoop-p11 kernel: [74767.673021] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:49:55.637216+00:00 bl-cassoop-p11 kernel: [74767.673022] Stack:
2014-01-18T13:49:55.637244+00:00 bl-cassoop-p11 kernel: [74767.673023]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:49:55.637273+00:00 bl-cassoop-p11 kernel: [74767.673029]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:49:55.637301+00:00 bl-cassoop-p11 kernel: [74767.673033]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:49:55.637329+00:00 bl-cassoop-p11 kernel: [74767.673037] Call Trace:
2014-01-18T13:49:55.637357+00:00 bl-cassoop-p11 kernel: [74767.673041]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.637386+00:00 bl-cassoop-p11 kernel: [74767.673045]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.637414+00:00 bl-cassoop-p11 kernel: [74767.673048]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:49:55.637442+00:00 bl-cassoop-p11 kernel: [74767.673051]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:49:55.637470+00:00 bl-cassoop-p11 kernel: [74767.673055]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:49:55.637499+00:00 bl-cassoop-p11 kernel: [74767.673058]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:49:55.637528+00:00 bl-cassoop-p11 kernel: [74767.673063]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:49:55.637556+00:00 bl-cassoop-p11 kernel: [74767.673066]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:49:55.637584+00:00 bl-cassoop-p11 kernel: [74767.673069]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:49:55.637613+00:00 bl-cassoop-p11 kernel: [74767.673072]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:49:55.637649+00:00 bl-cassoop-p11 kernel: [74767.673075]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:49:55.637678+00:00 bl-cassoop-p11 kernel: [74767.673078]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:49:55.637712+00:00 bl-cassoop-p11 kernel: [74767.673081]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:49:55.637744+00:00 bl-cassoop-p11 kernel: [74767.673086]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:55.637772+00:00 bl-cassoop-p11 kernel: [74767.673089]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:55.637801+00:00 bl-cassoop-p11 kernel: [74767.673092]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:55.637836+00:00 bl-cassoop-p11 kernel: [74767.673096]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:49:55.637865+00:00 bl-cassoop-p11 kernel: [74767.673099]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:49:55.637893+00:00 bl-cassoop-p11 kernel: [74767.673103]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:49:55.637921+00:00 bl-cassoop-p11 kernel: [74767.673106]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:49:55.637950+00:00 bl-cassoop-p11 kernel: [74767.673109]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:49:55.637954+00:00 bl-cassoop-p11 kernel: [74767.673112]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:55.637955+00:00 bl-cassoop-p11 kernel: [74767.673117]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:55.637958+00:00 bl-cassoop-p11 kernel: [74767.673118] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:49:55.648709+00:00 bl-cassoop-p11 kernel: [74767.682232] BUG: soft lockup - CPU#18 stuck for 22s! [kswapd0:283]
2014-01-18T13:49:55.648834+00:00 bl-cassoop-p11 kernel: [74767.684985] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.648839+00:00 bl-cassoop-p11 kernel: [74767.685007] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.648841+00:00 bl-cassoop-p11 kernel: [74767.685009] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.648843+00:00 bl-cassoop-p11 kernel: [74767.685011] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:49:55.648845+00:00 bl-cassoop-p11 kernel: [74767.685012] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:49:55.648847+00:00 bl-cassoop-p11 kernel: [74767.685015] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:49:55.648848+00:00 bl-cassoop-p11 kernel: [74767.685017] RAX: 0000000000000282 RBX: 0000000000000000 RCX: 0000000000000001
2014-01-18T13:49:55.648850+00:00 bl-cassoop-p11 kernel: [74767.685019] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:49:55.648851+00:00 bl-cassoop-p11 kernel: [74767.685020] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:49:55.648855+00:00 bl-cassoop-p11 kernel: [74767.685022] R10: 0000000000000001 R11: 0000000000000400 R12: ffffffff810a2fcc
2014-01-18T13:49:55.648857+00:00 bl-cassoop-p11 kernel: [74767.685024] R13: ffff882ff8361698 R14: ffff8817f2191200 R15: ffff882ff8361648
2014-01-18T13:49:55.648859+00:00 bl-cassoop-p11 kernel: [74767.685026] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.648861+00:00 bl-cassoop-p11 kernel: [74767.685028] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.648863+00:00 bl-cassoop-p11 kernel: [74767.685029] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:55.648864+00:00 bl-cassoop-p11 kernel: [74767.685031] Stack:
2014-01-18T13:49:55.648866+00:00 bl-cassoop-p11 kernel: [74767.685032]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:49:55.648868+00:00 bl-cassoop-p11 kernel: [74767.685037]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:49:55.648870+00:00 bl-cassoop-p11 kernel: [74767.685041]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:49:55.648871+00:00 bl-cassoop-p11 kernel: [74767.685046] Call Trace:
2014-01-18T13:49:55.648873+00:00 bl-cassoop-p11 kernel: [74767.685050]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.648875+00:00 bl-cassoop-p11 kernel: [74767.685053]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:55.648877+00:00 bl-cassoop-p11 kernel: [74767.685056]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.648879+00:00 bl-cassoop-p11 kernel: [74767.685060]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.648881+00:00 bl-cassoop-p11 kernel: [74767.685063]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:55.648883+00:00 bl-cassoop-p11 kernel: [74767.685067]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.648885+00:00 bl-cassoop-p11 kernel: [74767.685070]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.648887+00:00 bl-cassoop-p11 kernel: [74767.685074]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:55.648889+00:00 bl-cassoop-p11 kernel: [74767.685077]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:49:55.648890+00:00 bl-cassoop-p11 kernel: [74767.685081]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:49:55.648893+00:00 bl-cassoop-p11 kernel: [74767.685084]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:49:55.648895+00:00 bl-cassoop-p11 kernel: [74767.685088]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:49:55.648896+00:00 bl-cassoop-p11 kernel: [74767.685091]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:49:55.648899+00:00 bl-cassoop-p11 kernel: [74767.685095]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:49:55.648901+00:00 bl-cassoop-p11 kernel: [74767.685098]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:49:55.648902+00:00 bl-cassoop-p11 kernel: [74767.685101]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:49:55.648905+00:00 bl-cassoop-p11 kernel: [74767.685105]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:55.648906+00:00 bl-cassoop-p11 kernel: [74767.685109]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:55.648908+00:00 bl-cassoop-p11 kernel: [74767.685112]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:55.648910+00:00 bl-cassoop-p11 kernel: [74767.685116]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:49:55.648913+00:00 bl-cassoop-p11 kernel: [74767.685119]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:55.648915+00:00 bl-cassoop-p11 kernel: [74767.685123]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:49:55.648917+00:00 bl-cassoop-p11 kernel: [74767.685126]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:49:55.648919+00:00 bl-cassoop-p11 kernel: [74767.685129]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:49:55.648921+00:00 bl-cassoop-p11 kernel: [74767.685133]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:49:55.648922+00:00 bl-cassoop-p11 kernel: [74767.685136]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:49:55.648924+00:00 bl-cassoop-p11 kernel: [74767.685140]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:49:55.648925+00:00 bl-cassoop-p11 kernel: [74767.685143]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.648927+00:00 bl-cassoop-p11 kernel: [74767.685147]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:49:55.648929+00:00 bl-cassoop-p11 kernel: [74767.685151]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:49:55.648935+00:00 bl-cassoop-p11 kernel: [74767.685152] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:49:55.672559+00:00 bl-cassoop-p11 kernel: [74767.706239] BUG: soft lockup - CPU#20 stuck for 22s! [jsvc:42019]
2014-01-18T13:49:55.672568+00:00 bl-cassoop-p11 kernel: [74767.708964] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.672571+00:00 bl-cassoop-p11 kernel: [74767.708984] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.672573+00:00 bl-cassoop-p11 kernel: [74767.708986] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.672575+00:00 bl-cassoop-p11 kernel: [74767.708987] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:49:55.672577+00:00 bl-cassoop-p11 kernel: [74767.708988] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:49:55.672578+00:00 bl-cassoop-p11 kernel: [74767.708992] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:49:55.672580+00:00 bl-cassoop-p11 kernel: [74767.708993] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa387c0
2014-01-18T13:49:55.672581+00:00 bl-cassoop-p11 kernel: [74767.708994] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:49:55.672583+00:00 bl-cassoop-p11 kernel: [74767.708995] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:49:55.672585+00:00 bl-cassoop-p11 kernel: [74767.708996] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:49:55.672586+00:00 bl-cassoop-p11 kernel: [74767.708997] R13: ffff8801111b3c48 R14: 0000000000000292 R15: ffff8801111b3c78
2014-01-18T13:49:55.672588+00:00 bl-cassoop-p11 kernel: [74767.708998] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.672589+00:00 bl-cassoop-p11 kernel: [74767.709000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.672590+00:00 bl-cassoop-p11 kernel: [74767.709001] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:49:55.672591+00:00 bl-cassoop-p11 kernel: [74767.709002] Stack:
2014-01-18T13:49:55.672593+00:00 bl-cassoop-p11 kernel: [74767.709003]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:49:55.672594+00:00 bl-cassoop-p11 kernel: [74767.709006]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:49:55.672596+00:00 bl-cassoop-p11 kernel: [74767.709009]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:49:55.672597+00:00 bl-cassoop-p11 kernel: [74767.709012] Call Trace:
2014-01-18T13:49:55.672599+00:00 bl-cassoop-p11 kernel: [74767.709015]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.672600+00:00 bl-cassoop-p11 kernel: [74767.709018]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:55.672602+00:00 bl-cassoop-p11 kernel: [74767.709021]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:49:55.672604+00:00 bl-cassoop-p11 kernel: [74767.709024]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:49:55.672605+00:00 bl-cassoop-p11 kernel: [74767.709026]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:49:55.672607+00:00 bl-cassoop-p11 kernel: [74767.709028]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:49:55.672608+00:00 bl-cassoop-p11 kernel: [74767.709031]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:49:55.672610+00:00 bl-cassoop-p11 kernel: [74767.709033]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:49:55.672611+00:00 bl-cassoop-p11 kernel: [74767.709036]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:49:55.672614+00:00 bl-cassoop-p11 kernel: [74767.709037] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:49:55.756621+00:00 bl-cassoop-p11 kernel: [74767.790261] BUG: soft lockup - CPU#27 stuck for 22s! [java:42434]
2014-01-18T13:49:55.756630+00:00 bl-cassoop-p11 kernel: [74767.792956] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.756633+00:00 bl-cassoop-p11 kernel: [74767.792972] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.756634+00:00 bl-cassoop-p11 kernel: [74767.792974] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.756636+00:00 bl-cassoop-p11 kernel: [74767.792975] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:49:55.756638+00:00 bl-cassoop-p11 kernel: [74767.792976] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:55.756640+00:00 bl-cassoop-p11 kernel: [74767.792979] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:49:55.756641+00:00 bl-cassoop-p11 kernel: [74767.792980] RAX: 0000000000000030 RBX: ffffffff810a3789 RCX: 0000000000000010
2014-01-18T13:49:55.756642+00:00 bl-cassoop-p11 kernel: [74767.792981] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:49:55.756644+00:00 bl-cassoop-p11 kernel: [74767.792983] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:49:55.756646+00:00 bl-cassoop-p11 kernel: [74767.792984] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:49:55.756647+00:00 bl-cassoop-p11 kernel: [74767.792985] R13: 000000000000000e R14: ffffffff810a108e R15: ffff881887d733b8
2014-01-18T13:49:55.756649+00:00 bl-cassoop-p11 kernel: [74767.792986] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.756650+00:00 bl-cassoop-p11 kernel: [74767.792988] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.756652+00:00 bl-cassoop-p11 kernel: [74767.792989] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:49:55.756653+00:00 bl-cassoop-p11 kernel: [74767.792990] Stack:
2014-01-18T13:49:55.756654+00:00 bl-cassoop-p11 kernel: [74767.792990]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:49:55.756656+00:00 bl-cassoop-p11 kernel: [74767.792994]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:49:55.756657+00:00 bl-cassoop-p11 kernel: [74767.792997]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:49:55.756658+00:00 bl-cassoop-p11 kernel: [74767.793000] Call Trace:
2014-01-18T13:49:55.756660+00:00 bl-cassoop-p11 kernel: [74767.793003]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.756662+00:00 bl-cassoop-p11 kernel: [74767.793005]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:55.756663+00:00 bl-cassoop-p11 kernel: [74767.793008]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.756665+00:00 bl-cassoop-p11 kernel: [74767.793010]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.756666+00:00 bl-cassoop-p11 kernel: [74767.793012]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:55.756668+00:00 bl-cassoop-p11 kernel: [74767.793015]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.756670+00:00 bl-cassoop-p11 kernel: [74767.793017]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:49:55.756671+00:00 bl-cassoop-p11 kernel: [74767.793020]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:49:55.756673+00:00 bl-cassoop-p11 kernel: [74767.793022]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:49:55.756675+00:00 bl-cassoop-p11 kernel: [74767.793026]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:49:55.756676+00:00 bl-cassoop-p11 kernel: [74767.793028]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:49:55.756678+00:00 bl-cassoop-p11 kernel: [74767.793030]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:49:55.756679+00:00 bl-cassoop-p11 kernel: [74767.793033]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:49:55.756681+00:00 bl-cassoop-p11 kernel: [74767.793036]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:49:55.756683+00:00 bl-cassoop-p11 kernel: [74767.793038]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:49:55.756684+00:00 bl-cassoop-p11 kernel: [74767.793041]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:49:55.756686+00:00 bl-cassoop-p11 kernel: [74767.793044]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:49:55.756687+00:00 bl-cassoop-p11 kernel: [74767.793046]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:49:55.756689+00:00 bl-cassoop-p11 kernel: [74767.793050]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:49:55.756691+00:00 bl-cassoop-p11 kernel: [74767.793052]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:49:55.756692+00:00 bl-cassoop-p11 kernel: [74767.793055]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:49:55.756694+00:00 bl-cassoop-p11 kernel: [74767.793057]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:49:55.756696+00:00 bl-cassoop-p11 kernel: [74767.793060]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:49:55.756697+00:00 bl-cassoop-p11 kernel: [74767.793063]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:49:55.756699+00:00 bl-cassoop-p11 kernel: [74767.793066]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:49:55.756701+00:00 bl-cassoop-p11 kernel: [74767.793068]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:49:55.756702+00:00 bl-cassoop-p11 kernel: [74767.793071]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:49:55.756704+00:00 bl-cassoop-p11 kernel: [74767.793073]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:49:55.756706+00:00 bl-cassoop-p11 kernel: [74767.793076]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:49:55.756708+00:00 bl-cassoop-p11 kernel: [74767.793078]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:49:55.756709+00:00 bl-cassoop-p11 kernel: [74767.793080]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:49:55.756711+00:00 bl-cassoop-p11 kernel: [74767.793083]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:49:55.756723+00:00 bl-cassoop-p11 kernel: [74767.793085]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:49:55.756725+00:00 bl-cassoop-p11 kernel: [74767.793087]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:49:55.756727+00:00 bl-cassoop-p11 kernel: [74767.793093]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:49:55.756729+00:00 bl-cassoop-p11 kernel: [74767.793095]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:49:55.756731+00:00 bl-cassoop-p11 kernel: [74767.793098]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:49:55.756732+00:00 bl-cassoop-p11 kernel: [74767.793100]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:49:55.756734+00:00 bl-cassoop-p11 kernel: [74767.793103]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:49:55.756736+00:00 bl-cassoop-p11 kernel: [74767.793105]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:49:55.756737+00:00 bl-cassoop-p11 kernel: [74767.793107]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:49:55.756740+00:00 bl-cassoop-p11 kernel: [74767.793108] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:49:55.792621+00:00 bl-cassoop-p11 kernel: [74767.826286] BUG: soft lockup - CPU#30 stuck for 22s! [java:36244]
2014-01-18T13:49:55.792630+00:00 bl-cassoop-p11 kernel: [74767.828962] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:49:55.792633+00:00 bl-cassoop-p11 kernel: [74767.828984] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:49:55.792635+00:00 bl-cassoop-p11 kernel: [74767.828986] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:49:55.792636+00:00 bl-cassoop-p11 kernel: [74767.828987] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:49:55.792639+00:00 bl-cassoop-p11 kernel: [74767.828989] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:49:55.792640+00:00 bl-cassoop-p11 kernel: [74767.828992] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:49:55.792642+00:00 bl-cassoop-p11 kernel: [74767.828994] RAX: 0000000000000286 RBX: ffff88187fffcb08 RCX: 0000000000000001
2014-01-18T13:49:55.792643+00:00 bl-cassoop-p11 kernel: [74767.828996] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:49:55.792645+00:00 bl-cassoop-p11 kernel: [74767.828997] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:49:55.792646+00:00 bl-cassoop-p11 kernel: [74767.828999] R10: 00000000000035de R11: 0000000000000001 R12: 0000000000000000
2014-01-18T13:49:55.792648+00:00 bl-cassoop-p11 kernel: [74767.829001] R13: ffff88187fffcb00 R14: ffff88187fffcb08 R15: ffffffff00000000
2014-01-18T13:49:55.792649+00:00 bl-cassoop-p11 kernel: [74767.829003] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:49:55.792651+00:00 bl-cassoop-p11 kernel: [74767.829004] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:49:55.792652+00:00 bl-cassoop-p11 kernel: [74767.829006] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:49:55.792653+00:00 bl-cassoop-p11 kernel: [74767.829007] Stack:
2014-01-18T13:49:55.792655+00:00 bl-cassoop-p11 kernel: [74767.829008]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:49:55.792656+00:00 bl-cassoop-p11 kernel: [74767.829014]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:49:55.792658+00:00 bl-cassoop-p11 kernel: [74767.829019]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:49:55.792659+00:00 bl-cassoop-p11 kernel: [74767.829023] Call Trace:
2014-01-18T13:49:55.792660+00:00 bl-cassoop-p11 kernel: [74767.829027]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.792662+00:00 bl-cassoop-p11 kernel: [74767.829030]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:49:55.792664+00:00 bl-cassoop-p11 kernel: [74767.829033]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.792665+00:00 bl-cassoop-p11 kernel: [74767.829037]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:49:55.792667+00:00 bl-cassoop-p11 kernel: [74767.829040]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:49:55.792668+00:00 bl-cassoop-p11 kernel: [74767.829043]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:49:55.792670+00:00 bl-cassoop-p11 kernel: [74767.829046]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:49:55.792672+00:00 bl-cassoop-p11 kernel: [74767.829050]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:49:55.792673+00:00 bl-cassoop-p11 kernel: [74767.829054]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:49:55.792675+00:00 bl-cassoop-p11 kernel: [74767.829057]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:49:55.792676+00:00 bl-cassoop-p11 kernel: [74767.829060]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:49:55.792678+00:00 bl-cassoop-p11 kernel: [74767.829063]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:49:55.792680+00:00 bl-cassoop-p11 kernel: [74767.829066]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:49:55.792681+00:00 bl-cassoop-p11 kernel: [74767.829069]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:49:55.792683+00:00 bl-cassoop-p11 kernel: [74767.829072]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:49:55.792684+00:00 bl-cassoop-p11 kernel: [74767.829076]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:49:55.792686+00:00 bl-cassoop-p11 kernel: [74767.829079]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:49:55.792687+00:00 bl-cassoop-p11 kernel: [74767.829082]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:49:55.792689+00:00 bl-cassoop-p11 kernel: [74767.829085]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:49:55.792690+00:00 bl-cassoop-p11 kernel: [74767.829088]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:49:55.792692+00:00 bl-cassoop-p11 kernel: [74767.829092]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:49:55.792693+00:00 bl-cassoop-p11 kernel: [74767.829095]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:49:55.792695+00:00 bl-cassoop-p11 kernel: [74767.829099]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:49:55.792697+00:00 bl-cassoop-p11 kernel: [74767.829102]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:49:55.792698+00:00 bl-cassoop-p11 kernel: [74767.829105]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:49:55.792700+00:00 bl-cassoop-p11 kernel: [74767.829109]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:49:55.792701+00:00 bl-cassoop-p11 kernel: [74767.829113]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:49:55.792703+00:00 bl-cassoop-p11 kernel: [74767.829116]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:49:55.792705+00:00 bl-cassoop-p11 kernel: [74767.829117] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:50:23.443401+00:00 bl-cassoop-p11 kernel: [74789.470105] ------------[ cut here ]------------
2014-01-18T13:50:23.443410+00:00 bl-cassoop-p11 kernel: [74789.470113] WARNING: CPU: 5 PID: 2544 at /build/buildd/linux-lts-saucy-3.11.0/kernel/watchdog.c:245 watchdog_overflow_callback+0x9a/0xc0()
2014-01-18T13:50:23.443413+00:00 bl-cassoop-p11 kernel: [74789.470114] Watchdog detected hard LOCKUP on cpu 5
2014-01-18T13:50:23.443417+00:00 bl-cassoop-p11 kernel: [74789.470116] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.443419+00:00 bl-cassoop-p11 kernel: [74789.470134] CPU: 5 PID: 2544 Comm: sshd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.443420+00:00 bl-cassoop-p11 kernel: [74789.470135] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.443422+00:00 bl-cassoop-p11 kernel: [74789.470136]  00000000000000f5 ffff88301f247ba8 ffffffff8173bc0e 0000000000000007
2014-01-18T13:50:23.443423+00:00 bl-cassoop-p11 kernel: [74789.470141]  ffff88301f247bf8 ffff88301f247be8 ffffffff810653ac 00000000000123f3
2014-01-18T13:50:23.443425+00:00 bl-cassoop-p11 kernel: [74789.470144]  ffff882ffa458000 0000000000000000 ffff88301f247d20 0000000000000000
2014-01-18T13:50:23.443426+00:00 bl-cassoop-p11 kernel: [74789.470147] Call Trace:
2014-01-18T13:50:23.443428+00:00 bl-cassoop-p11 kernel: [74789.470148]  <NMI>  [<ffffffff8173bc0e>] dump_stack+0x46/0x58
2014-01-18T13:50:23.443430+00:00 bl-cassoop-p11 kernel: [74789.470156]  [<ffffffff810653ac>] warn_slowpath_common+0x8c/0xc0
2014-01-18T13:50:23.443432+00:00 bl-cassoop-p11 kernel: [74789.470158]  [<ffffffff81065496>] warn_slowpath_fmt+0x46/0x50
2014-01-18T13:50:23.443433+00:00 bl-cassoop-p11 kernel: [74789.470160]  [<ffffffff810fbf8a>] watchdog_overflow_callback+0x9a/0xc0
2014-01-18T13:50:23.443435+00:00 bl-cassoop-p11 kernel: [74789.470164]  [<ffffffff8113f9cc>] __perf_event_overflow+0x9c/0x220
2014-01-18T13:50:23.443437+00:00 bl-cassoop-p11 kernel: [74789.470168]  [<ffffffff81029828>] ? x86_perf_event_set_period+0xd8/0x150
2014-01-18T13:50:23.443439+00:00 bl-cassoop-p11 kernel: [74789.470172]  [<ffffffff811402d4>] perf_event_overflow+0x14/0x20
2014-01-18T13:50:23.443440+00:00 bl-cassoop-p11 kernel: [74789.470175]  [<ffffffff81030ff6>] intel_pmu_handle_irq+0x1a6/0x2a0
2014-01-18T13:50:23.443442+00:00 bl-cassoop-p11 kernel: [74789.470178]  [<ffffffff811813d1>] ? unmap_kernel_range_noflush+0x11/0x20
2014-01-18T13:50:23.443444+00:00 bl-cassoop-p11 kernel: [74789.470182]  [<ffffffff81426163>] ? ghes_copy_tofrom_phys+0x113/0x210
2014-01-18T13:50:23.443445+00:00 bl-cassoop-p11 kernel: [74789.470186]  [<ffffffff817496a4>] perf_event_nmi_handler+0x34/0x60
2014-01-18T13:50:23.443447+00:00 bl-cassoop-p11 kernel: [74789.470188]  [<ffffffff81748caa>] nmi_handle.isra.3+0x8a/0x1a0
2014-01-18T13:50:23.443449+00:00 bl-cassoop-p11 kernel: [74789.470191]  [<ffffffff81427160>] ? ghes_print_estatus.constprop.10+0x70/0x70
2014-01-18T13:50:23.443450+00:00 bl-cassoop-p11 kernel: [74789.470194]  [<ffffffff81748f39>] default_do_nmi+0xe9/0x240
2014-01-18T13:50:23.443452+00:00 bl-cassoop-p11 kernel: [74789.470196]  [<ffffffff81749120>] do_nmi+0x90/0xd0
2014-01-18T13:50:23.443453+00:00 bl-cassoop-p11 kernel: [74789.470199]  [<ffffffff817481c1>] end_repeat_nmi+0x1e/0x2e
2014-01-18T13:50:23.443455+00:00 bl-cassoop-p11 kernel: [74789.470203]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:50:23.443456+00:00 bl-cassoop-p11 kernel: [74789.470205]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:50:23.443458+00:00 bl-cassoop-p11 kernel: [74789.470207]  [<ffffffff813817d3>] ? __write_lock_failed+0x13/0x20
2014-01-18T13:50:23.443460+00:00 bl-cassoop-p11 kernel: [74789.470208]  <<EOE>>  [<ffffffff8174787e>] _raw_write_lock_irq+0x1e/0x20
2014-01-18T13:50:23.443461+00:00 bl-cassoop-p11 kernel: [74789.470213]  [<ffffffff81063fda>] copy_process.part.24+0xa2a/0xea0
2014-01-18T13:50:23.443463+00:00 bl-cassoop-p11 kernel: [74789.470217]  [<ffffffff812f0e16>] ? security_file_alloc+0x16/0x20
2014-01-18T13:50:23.443465+00:00 bl-cassoop-p11 kernel: [74789.470220]  [<ffffffff811b5910>] ? get_empty_filp+0x90/0x1b0
2014-01-18T13:50:23.443466+00:00 bl-cassoop-p11 kernel: [74789.470222]  [<ffffffff810644d0>] copy_process+0x80/0x90
2014-01-18T13:50:23.443468+00:00 bl-cassoop-p11 kernel: [74789.470224]  [<ffffffff8106464c>] do_fork+0x9c/0x220
2014-01-18T13:50:23.443469+00:00 bl-cassoop-p11 kernel: [74789.470226]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:23.443471+00:00 bl-cassoop-p11 kernel: [74789.470229]  [<ffffffff811d1122>] ? __fd_install+0x52/0x70
2014-01-18T13:50:23.443472+00:00 bl-cassoop-p11 kernel: [74789.470231]  [<ffffffff81064856>] SyS_clone+0x16/0x20
2014-01-18T13:50:23.443474+00:00 bl-cassoop-p11 kernel: [74789.470234]  [<ffffffff81750cb9>] stub_clone+0x69/0x90
2014-01-18T13:50:23.443475+00:00 bl-cassoop-p11 kernel: [74789.470236]  [<ffffffff8175095d>] ? system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:23.443477+00:00 bl-cassoop-p11 kernel: [74789.470237] ---[ end trace 51e5206791572f03 ]---
2014-01-18T13:50:23.443478+00:00 bl-cassoop-p11 kernel: [74795.484371] BUG: soft lockup - CPU#0 stuck for 22s! [java:42506]
2014-01-18T13:50:23.443480+00:00 bl-cassoop-p11 kernel: [74795.484374] BUG: soft lockup - CPU#1 stuck for 22s! [collectd:5830]
2014-01-18T13:50:23.443484+00:00 bl-cassoop-p11 kernel: [74795.484392] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.443486+00:00 bl-cassoop-p11 kernel: [74795.484394] CPU: 1 PID: 5830 Comm: collectd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.443487+00:00 bl-cassoop-p11 kernel: [74795.484395] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.443489+00:00 bl-cassoop-p11 kernel: [74795.484396] task: ffff882ff0912ee0 ti: ffff882ffa6f6000 task.ti: ffff882ffa6f6000
2014-01-18T13:50:23.443533+00:00 bl-cassoop-p11 kernel: [74795.484401] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:50:23.443535+00:00 bl-cassoop-p11 kernel: [74795.484402] RSP: 0018:ffff882ffa6f7d08  EFLAGS: 00000202
2014-01-18T13:50:23.443537+00:00 bl-cassoop-p11 kernel: [74795.484403] RAX: 0000000000000286 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:50:23.443538+00:00 bl-cassoop-p11 kernel: [74795.484403] RDX: ffff88181fa38680 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:50:23.443540+00:00 bl-cassoop-p11 kernel: [74795.484404] RBP: ffff882ffa6f7d48 R08: 0000000000000000 R09: 000000000000730e
2014-01-18T13:50:23.443541+00:00 bl-cassoop-p11 kernel: [74795.484405] R10: 0000000000003708 R11: 0000000000000293 R12: 0000000000000000
2014-01-18T13:50:23.443543+00:00 bl-cassoop-p11 kernel: [74795.484406] R13: 00007f2a9217e000 R14: ffff88307fff9d80 R15: ffff882ffa6f7f24
2014-01-18T13:50:23.443544+00:00 bl-cassoop-p11 kernel: [74795.484407] FS:  00007f2a8a89f700(0000) GS:ffff88301f200000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.443546+00:00 bl-cassoop-p11 kernel: [74795.484408] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.443547+00:00 bl-cassoop-p11 kernel: [74795.484409] CR2: 00007f2a92182000 CR3: 0000002ff0b0e000 CR4: 00000000000407e0
2014-01-18T13:50:23.443548+00:00 bl-cassoop-p11 kernel: [74795.484410] Stack:
2014-01-18T13:50:23.443550+00:00 bl-cassoop-p11 kernel: [74795.484416]  ffff882ff7817000 ffff88181fa386a8 00000007f2a9217e 0000000000000002
2014-01-18T13:50:23.443551+00:00 bl-cassoop-p11 kernel: [74795.484419]  ffffffff8139a810 0000000000000001 ffffffff81d03da0 00007f2a92182000
2014-01-18T13:50:23.443553+00:00 bl-cassoop-p11 kernel: [74795.484422]  ffff882ffa6f7db8 ffffffff810cb5a5 ffff882ff0912ee0 0000000000000001
2014-01-18T13:50:23.443554+00:00 bl-cassoop-p11 kernel: [74795.484423] Call Trace:
2014-01-18T13:50:23.443556+00:00 bl-cassoop-p11 kernel: [74795.484429]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:50:23.443557+00:00 bl-cassoop-p11 kernel: [74795.484431]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:23.443559+00:00 bl-cassoop-p11 kernel: [74795.484433]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:50:23.443561+00:00 bl-cassoop-p11 kernel: [74795.484437]  [<ffffffff8139ab4e>] rdmsr_on_cpu+0x4e/0x70
2014-01-18T13:50:23.443562+00:00 bl-cassoop-p11 kernel: [74795.484441]  [<ffffffffa001b62b>] show_temp+0xab/0xf0 [coretemp]
2014-01-18T13:50:23.443564+00:00 bl-cassoop-p11 kernel: [74795.484444]  [<ffffffff81490f20>] dev_attr_show+0x20/0x60
2014-01-18T13:50:23.443565+00:00 bl-cassoop-p11 kernel: [74795.484448]  [<ffffffff8114d37e>] ? __get_free_pages+0xe/0x40
2014-01-18T13:50:23.443567+00:00 bl-cassoop-p11 kernel: [74795.484451]  [<ffffffff81227e86>] fill_read_buffer.isra.9+0x66/0xf0
2014-01-18T13:50:23.443569+00:00 bl-cassoop-p11 kernel: [74795.484454]  [<ffffffff81227fc0>] sysfs_read_file+0xb0/0xf0
2014-01-18T13:50:23.443570+00:00 bl-cassoop-p11 kernel: [74795.484456]  [<ffffffff811b42e4>] vfs_read+0xb4/0x180
2014-01-18T13:50:23.443572+00:00 bl-cassoop-p11 kernel: [74795.484459]  [<ffffffff811b4562>] SyS_read+0x52/0xa0
2014-01-18T13:50:23.443574+00:00 bl-cassoop-p11 kernel: [74795.484462]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:23.443576+00:00 bl-cassoop-p11 kernel: [74795.484481] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:50:23.443580+00:00 bl-cassoop-p11 kernel: [74795.489883] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.443582+00:00 bl-cassoop-p11 kernel: [74795.489906] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.443584+00:00 bl-cassoop-p11 kernel: [74795.489908] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.443586+00:00 bl-cassoop-p11 kernel: [74795.489910] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:50:23.443588+00:00 bl-cassoop-p11 kernel: [74795.489912] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.443590+00:00 bl-cassoop-p11 kernel: [74795.489916] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:50:23.443591+00:00 bl-cassoop-p11 kernel: [74795.489917] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa97868
2014-01-18T13:50:23.443593+00:00 bl-cassoop-p11 kernel: [74795.489919] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.443594+00:00 bl-cassoop-p11 kernel: [74795.489921] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:50:23.443596+00:00 bl-cassoop-p11 kernel: [74795.489922] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:50:23.443597+00:00 bl-cassoop-p11 kernel: [74795.489924] R13: ffff880045ccfc48 R14: 0000000000000292 R15: ffff880045ccfc78
2014-01-18T13:50:23.443599+00:00 bl-cassoop-p11 kernel: [74795.489926] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.443600+00:00 bl-cassoop-p11 kernel: [74795.489928] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.443602+00:00 bl-cassoop-p11 kernel: [74795.489930] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:50:23.443603+00:00 bl-cassoop-p11 kernel: [74795.489931] Stack:
2014-01-18T13:50:23.443605+00:00 bl-cassoop-p11 kernel: [74795.489932]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:50:23.443606+00:00 bl-cassoop-p11 kernel: [74795.489939]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:50:23.443608+00:00 bl-cassoop-p11 kernel: [74795.489944]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:50:23.443609+00:00 bl-cassoop-p11 kernel: [74795.489948] Call Trace:
2014-01-18T13:50:23.443611+00:00 bl-cassoop-p11 kernel: [74795.489953]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.443612+00:00 bl-cassoop-p11 kernel: [74795.489957]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:23.443621+00:00 bl-cassoop-p11 kernel: [74795.489960]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:50:23.443623+00:00 bl-cassoop-p11 kernel: [74795.489964]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:50:23.443625+00:00 bl-cassoop-p11 kernel: [74795.489967]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:50:23.443626+00:00 bl-cassoop-p11 kernel: [74795.489970]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:50:23.443628+00:00 bl-cassoop-p11 kernel: [74795.489974]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:50:23.443629+00:00 bl-cassoop-p11 kernel: [74795.489977]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:50:23.443631+00:00 bl-cassoop-p11 kernel: [74795.489981]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:23.443634+00:00 bl-cassoop-p11 kernel: [74795.489982] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.552670+00:00 bl-cassoop-p11 kernel: [74795.596409] BUG: soft lockup - CPU#10 stuck for 22s! [java:42435]
2014-01-18T13:50:23.552680+00:00 bl-cassoop-p11 kernel: [74795.599204] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.552683+00:00 bl-cassoop-p11 kernel: [74795.599221] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.552685+00:00 bl-cassoop-p11 kernel: [74795.599222] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.552686+00:00 bl-cassoop-p11 kernel: [74795.599224] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:50:23.552689+00:00 bl-cassoop-p11 kernel: [74795.599225] RIP: 0010:[<ffffffff81731068>]  [<ffffffff81731068>] oom_kill_process.part.6+0xae/0x2cf
2014-01-18T13:50:23.552690+00:00 bl-cassoop-p11 kernel: [74795.599229] RSP: 0000:ffff8819e9a81a28  EFLAGS: 00000206
2014-01-18T13:50:23.552692+00:00 bl-cassoop-p11 kernel: [74795.599230] RAX: ffff8818979f6150 RBX: 00000000000000d0 RCX: 0000000000000006
2014-01-18T13:50:23.552693+00:00 bl-cassoop-p11 kernel: [74795.599231] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffffffff81c06040
2014-01-18T13:50:23.552695+00:00 bl-cassoop-p11 kernel: [74795.599232] RBP: ffff8819e9a81a88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:50:23.552696+00:00 bl-cassoop-p11 kernel: [74795.599233] R10: 0000000000000e32 R11: 0000000000005000 R12: ffff882ff044e3f0
2014-01-18T13:50:23.552698+00:00 bl-cassoop-p11 kernel: [74795.599234] R13: 0000000000000005 R14: 0000000000000246 R15: 0000000028e74000
2014-01-18T13:50:23.552700+00:00 bl-cassoop-p11 kernel: [74795.599236] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.552701+00:00 bl-cassoop-p11 kernel: [74795.599237] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.552702+00:00 bl-cassoop-p11 kernel: [74795.599238] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:50:23.552703+00:00 bl-cassoop-p11 kernel: [74795.599239] Stack:
2014-01-18T13:50:23.552705+00:00 bl-cassoop-p11 kernel: [74795.599240]  ffff8819e9a81a38 ffff882ff044ddc0 000000000114eccd ffffc90028e74000
2014-01-18T13:50:23.552706+00:00 bl-cassoop-p11 kernel: [74795.599244]  ffff8819e9a81a88 ffff8818979f6150 ffff882ff5c55dc0 000000000114eccd
2014-01-18T13:50:23.552708+00:00 bl-cassoop-p11 kernel: [74795.599246]  0000000000000000 ffffc90028e74000 ffff882ff0b86a00 ffff882ff044ddc0
2014-01-18T13:50:23.552709+00:00 bl-cassoop-p11 kernel: [74795.599249] Call Trace:
2014-01-18T13:50:23.552711+00:00 bl-cassoop-p11 kernel: [74795.599252]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:50:23.552712+00:00 bl-cassoop-p11 kernel: [74795.599255]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:50:23.552714+00:00 bl-cassoop-p11 kernel: [74795.599257]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:50:23.552716+00:00 bl-cassoop-p11 kernel: [74795.599259]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:50:23.552717+00:00 bl-cassoop-p11 kernel: [74795.599262]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:50:23.552719+00:00 bl-cassoop-p11 kernel: [74795.599265]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:50:23.552721+00:00 bl-cassoop-p11 kernel: [74795.599267]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:23.552722+00:00 bl-cassoop-p11 kernel: [74795.599270]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:23.552724+00:00 bl-cassoop-p11 kernel: [74795.599272]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:50:23.552726+00:00 bl-cassoop-p11 kernel: [74795.599275]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:50:23.552727+00:00 bl-cassoop-p11 kernel: [74795.599277]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:50:23.552729+00:00 bl-cassoop-p11 kernel: [74795.599280]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:23.552731+00:00 bl-cassoop-p11 kernel: [74795.599282]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:50:23.552732+00:00 bl-cassoop-p11 kernel: [74795.599284]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:23.552734+00:00 bl-cassoop-p11 kernel: [74795.599287]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:50:23.552735+00:00 bl-cassoop-p11 kernel: [74795.599289]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:23.552737+00:00 bl-cassoop-p11 kernel: [74795.599292]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:23.552738+00:00 bl-cassoop-p11 kernel: [74795.599294]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:50:23.552740+00:00 bl-cassoop-p11 kernel: [74795.599296]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:50:23.552742+00:00 bl-cassoop-p11 kernel: [74795.599298]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:23.552743+00:00 bl-cassoop-p11 kernel: [74795.599301]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:50:23.552745+00:00 bl-cassoop-p11 kernel: [74795.599303]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:23.552746+00:00 bl-cassoop-p11 kernel: [74795.599306]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:23.552749+00:00 bl-cassoop-p11 kernel: [74795.599307] Code: d1 e5 ff ff 48 89 df e8 e7 ff 91 ff 66 90 48 c7 c7 40 60 c0 81 4d 89 fe 45 31 ed e8 63 65 01 00 4c 89 7d a8 49 8b 9e 00 03 00 00 <4d> 8d a6 00 03 00 00 48 81 eb 10 03 00 00 eb 53 49 8b 87 a0 02
2014-01-18T13:50:23.576609+00:00 bl-cassoop-p11 kernel: [74795.620426] BUG: soft lockup - CPU#12 stuck for 22s! [jsvc:7242]
2014-01-18T13:50:23.576618+00:00 bl-cassoop-p11 kernel: [74795.623158] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.576621+00:00 bl-cassoop-p11 kernel: [74795.623182] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.576623+00:00 bl-cassoop-p11 kernel: [74795.623184] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.576624+00:00 bl-cassoop-p11 kernel: [74795.623185] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:50:23.576627+00:00 bl-cassoop-p11 kernel: [74795.623186] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.576628+00:00 bl-cassoop-p11 kernel: [74795.623196] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:50:23.576630+00:00 bl-cassoop-p11 kernel: [74795.623197] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa38680
2014-01-18T13:50:23.576631+00:00 bl-cassoop-p11 kernel: [74795.623198] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.576633+00:00 bl-cassoop-p11 kernel: [74795.623199] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:50:23.576634+00:00 bl-cassoop-p11 kernel: [74795.623200] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff8817f37ca000
2014-01-18T13:50:23.576636+00:00 bl-cassoop-p11 kernel: [74795.623201] R13: 0000000000000001 R14: 0000000000000292 R15: ffff8817f8e45938
2014-01-18T13:50:23.576637+00:00 bl-cassoop-p11 kernel: [74795.623203] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.576639+00:00 bl-cassoop-p11 kernel: [74795.623204] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.576640+00:00 bl-cassoop-p11 kernel: [74795.623205] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:50:23.576641+00:00 bl-cassoop-p11 kernel: [74795.623206] Stack:
2014-01-18T13:50:23.576643+00:00 bl-cassoop-p11 kernel: [74795.623207]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:50:23.576644+00:00 bl-cassoop-p11 kernel: [74795.623211]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:50:23.576646+00:00 bl-cassoop-p11 kernel: [74795.623214]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:50:23.576647+00:00 bl-cassoop-p11 kernel: [74795.623216] Call Trace:
2014-01-18T13:50:23.576649+00:00 bl-cassoop-p11 kernel: [74795.623220]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.576650+00:00 bl-cassoop-p11 kernel: [74795.623223]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.576652+00:00 bl-cassoop-p11 kernel: [74795.623225]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:23.576653+00:00 bl-cassoop-p11 kernel: [74795.623227]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:50:23.576655+00:00 bl-cassoop-p11 kernel: [74795.623230]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:50:23.576657+00:00 bl-cassoop-p11 kernel: [74795.623232]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:50:23.576658+00:00 bl-cassoop-p11 kernel: [74795.623234]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:50:23.576660+00:00 bl-cassoop-p11 kernel: [74795.623236]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:50:23.576661+00:00 bl-cassoop-p11 kernel: [74795.623238]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:50:23.576663+00:00 bl-cassoop-p11 kernel: [74795.623240]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:50:23.576665+00:00 bl-cassoop-p11 kernel: [74795.623243]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:50:23.576666+00:00 bl-cassoop-p11 kernel: [74795.623245]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:50:23.576668+00:00 bl-cassoop-p11 kernel: [74795.623247]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:50:23.576670+00:00 bl-cassoop-p11 kernel: [74795.623249]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:50:23.576671+00:00 bl-cassoop-p11 kernel: [74795.623251]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:23.576673+00:00 bl-cassoop-p11 kernel: [74795.623254]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:23.576675+00:00 bl-cassoop-p11 kernel: [74795.623256]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:23.576676+00:00 bl-cassoop-p11 kernel: [74795.623258]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:23.576679+00:00 bl-cassoop-p11 kernel: [74795.623259] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.588565+00:00 bl-cassoop-p11 kernel: [74795.632422] BUG: soft lockup - CPU#13 stuck for 22s! [kswapd1:284]
2014-01-18T13:50:23.588573+00:00 bl-cassoop-p11 kernel: [74795.635108] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.588576+00:00 bl-cassoop-p11 kernel: [74795.635124] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.588578+00:00 bl-cassoop-p11 kernel: [74795.635126] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.588579+00:00 bl-cassoop-p11 kernel: [74795.635127] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:50:23.588582+00:00 bl-cassoop-p11 kernel: [74795.635128] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.588583+00:00 bl-cassoop-p11 kernel: [74795.635131] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:50:23.588585+00:00 bl-cassoop-p11 kernel: [74795.635132] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa386a8
2014-01-18T13:50:23.588586+00:00 bl-cassoop-p11 kernel: [74795.635133] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.588588+00:00 bl-cassoop-p11 kernel: [74795.635134] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:50:23.588589+00:00 bl-cassoop-p11 kernel: [74795.635135] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000009
2014-01-18T13:50:23.588591+00:00 bl-cassoop-p11 kernel: [74795.635136] R13: ffffffff8105ae90 R14: 0000000000000292 R15: ffff882ff8363748
2014-01-18T13:50:23.588592+00:00 bl-cassoop-p11 kernel: [74795.635137] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.588594+00:00 bl-cassoop-p11 kernel: [74795.635138] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.588595+00:00 bl-cassoop-p11 kernel: [74795.635140] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:23.588596+00:00 bl-cassoop-p11 kernel: [74795.635140] Stack:
2014-01-18T13:50:23.588598+00:00 bl-cassoop-p11 kernel: [74795.635141]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:50:23.588599+00:00 bl-cassoop-p11 kernel: [74795.635145]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:50:23.588601+00:00 bl-cassoop-p11 kernel: [74795.635148]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:50:23.588602+00:00 bl-cassoop-p11 kernel: [74795.635151] Call Trace:
2014-01-18T13:50:23.588604+00:00 bl-cassoop-p11 kernel: [74795.635157]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.588605+00:00 bl-cassoop-p11 kernel: [74795.635160]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.588607+00:00 bl-cassoop-p11 kernel: [74795.635162]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:23.588609+00:00 bl-cassoop-p11 kernel: [74795.635165]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:50:23.588610+00:00 bl-cassoop-p11 kernel: [74795.635167]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:50:23.588612+00:00 bl-cassoop-p11 kernel: [74795.635170]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:50:23.588614+00:00 bl-cassoop-p11 kernel: [74795.635173]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:50:23.588615+00:00 bl-cassoop-p11 kernel: [74795.635175]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:50:23.588617+00:00 bl-cassoop-p11 kernel: [74795.635178]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:23.588618+00:00 bl-cassoop-p11 kernel: [74795.635180]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:23.588620+00:00 bl-cassoop-p11 kernel: [74795.635184]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:23.588622+00:00 bl-cassoop-p11 kernel: [74795.635186]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:23.588623+00:00 bl-cassoop-p11 kernel: [74795.635189]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:23.588625+00:00 bl-cassoop-p11 kernel: [74795.635191]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:50:23.588627+00:00 bl-cassoop-p11 kernel: [74795.635193]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:50:23.588628+00:00 bl-cassoop-p11 kernel: [74795.635197]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:50:23.588630+00:00 bl-cassoop-p11 kernel: [74795.635199]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:50:23.588631+00:00 bl-cassoop-p11 kernel: [74795.635202]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:50:23.588633+00:00 bl-cassoop-p11 kernel: [74795.635205]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:23.588634+00:00 bl-cassoop-p11 kernel: [74795.635208]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.588636+00:00 bl-cassoop-p11 kernel: [74795.635210]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:23.588638+00:00 bl-cassoop-p11 kernel: [74795.635212]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.588640+00:00 bl-cassoop-p11 kernel: [74795.635213] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.600707+00:00 bl-cassoop-p11 kernel: [74795.644425] BUG: soft lockup - CPU#14 stuck for 22s! [java:42064]
2014-01-18T13:50:23.600715+00:00 bl-cassoop-p11 kernel: [74795.647115] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.600718+00:00 bl-cassoop-p11 kernel: [74795.647137] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.600720+00:00 bl-cassoop-p11 kernel: [74795.647138] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.600721+00:00 bl-cassoop-p11 kernel: [74795.647140] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:50:23.600724+00:00 bl-cassoop-p11 kernel: [74795.647142] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.600725+00:00 bl-cassoop-p11 kernel: [74795.647146] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:50:23.600726+00:00 bl-cassoop-p11 kernel: [74795.647147] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa986d0
2014-01-18T13:50:23.600728+00:00 bl-cassoop-p11 kernel: [74795.647149] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.600730+00:00 bl-cassoop-p11 kernel: [74795.647151] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:50:23.600731+00:00 bl-cassoop-p11 kernel: [74795.647152] R10: 0000000000000004 R11: 0000000000000058 R12: ffffffff81049a3e
2014-01-18T13:50:23.600733+00:00 bl-cassoop-p11 kernel: [74795.647154] R13: ffff88195e365078 R14: 0000000000000292 R15: ffff88195e3650a8
2014-01-18T13:50:23.600734+00:00 bl-cassoop-p11 kernel: [74795.647156] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.600735+00:00 bl-cassoop-p11 kernel: [74795.647158] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.600737+00:00 bl-cassoop-p11 kernel: [74795.647159] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:50:23.600738+00:00 bl-cassoop-p11 kernel: [74795.647161] Stack:
2014-01-18T13:50:23.600739+00:00 bl-cassoop-p11 kernel: [74795.647162]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:50:23.600741+00:00 bl-cassoop-p11 kernel: [74795.647167]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:50:23.600743+00:00 bl-cassoop-p11 kernel: [74795.647171]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:50:23.600744+00:00 bl-cassoop-p11 kernel: [74795.647176] Call Trace:
2014-01-18T13:50:23.600745+00:00 bl-cassoop-p11 kernel: [74795.647180]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.600747+00:00 bl-cassoop-p11 kernel: [74795.647184]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.600748+00:00 bl-cassoop-p11 kernel: [74795.647187]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:23.600750+00:00 bl-cassoop-p11 kernel: [74795.647191]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:50:23.600752+00:00 bl-cassoop-p11 kernel: [74795.647194]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:50:23.600753+00:00 bl-cassoop-p11 kernel: [74795.647198]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:50:23.600755+00:00 bl-cassoop-p11 kernel: [74795.647201]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:50:23.600756+00:00 bl-cassoop-p11 kernel: [74795.647205]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:50:23.600758+00:00 bl-cassoop-p11 kernel: [74795.647209]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:23.600760+00:00 bl-cassoop-p11 kernel: [74795.647212]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:23.600761+00:00 bl-cassoop-p11 kernel: [74795.647216]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:23.600763+00:00 bl-cassoop-p11 kernel: [74795.647219]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:50:23.600764+00:00 bl-cassoop-p11 kernel: [74795.647223]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:50:23.600766+00:00 bl-cassoop-p11 kernel: [74795.647226]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:50:23.600767+00:00 bl-cassoop-p11 kernel: [74795.647230]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:50:23.600769+00:00 bl-cassoop-p11 kernel: [74795.647233]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:50:23.600771+00:00 bl-cassoop-p11 kernel: [74795.647237]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:50:23.600772+00:00 bl-cassoop-p11 kernel: [74795.647241]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:50:23.600774+00:00 bl-cassoop-p11 kernel: [74795.647244]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:23.600776+00:00 bl-cassoop-p11 kernel: [74795.647248]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:23.600778+00:00 bl-cassoop-p11 kernel: [74795.647252]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:50:23.600779+00:00 bl-cassoop-p11 kernel: [74795.647255]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:50:23.600781+00:00 bl-cassoop-p11 kernel: [74795.647258]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:50:23.600783+00:00 bl-cassoop-p11 kernel: [74795.647262]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:50:23.600784+00:00 bl-cassoop-p11 kernel: [74795.647265]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:50:23.600786+00:00 bl-cassoop-p11 kernel: [74795.647269]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:50:23.600787+00:00 bl-cassoop-p11 kernel: [74795.647272]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:50:23.600789+00:00 bl-cassoop-p11 kernel: [74795.647276]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:50:23.600791+00:00 bl-cassoop-p11 kernel: [74795.647279]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:50:23.600792+00:00 bl-cassoop-p11 kernel: [74795.647282]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:50:23.600794+00:00 bl-cassoop-p11 kernel: [74795.647285]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:50:23.600796+00:00 bl-cassoop-p11 kernel: [74795.647288]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:50:23.600812+00:00 bl-cassoop-p11 kernel: [74795.647292]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:50:23.600814+00:00 bl-cassoop-p11 kernel: [74795.647295]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:23.600815+00:00 bl-cassoop-p11 kernel: [74795.647299]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:50:23.600817+00:00 bl-cassoop-p11 kernel: [74795.647302]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:23.600819+00:00 bl-cassoop-p11 kernel: [74795.647305]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:50:23.600820+00:00 bl-cassoop-p11 kernel: [74795.647309]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:50:23.600822+00:00 bl-cassoop-p11 kernel: [74795.647312]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:50:23.600824+00:00 bl-cassoop-p11 kernel: [74795.647315]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:50:23.600826+00:00 bl-cassoop-p11 kernel: [74795.647318]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:23.600827+00:00 bl-cassoop-p11 kernel: [74795.647321]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:23.600829+00:00 bl-cassoop-p11 kernel: [74795.647325]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:23.600830+00:00 bl-cassoop-p11 kernel: [74795.647328]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:50:23.600832+00:00 bl-cassoop-p11 kernel: [74795.647331]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:50:23.600834+00:00 bl-cassoop-p11 kernel: [74795.647335]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:50:23.600835+00:00 bl-cassoop-p11 kernel: [74795.647338]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:23.600837+00:00 bl-cassoop-p11 kernel: [74795.647341]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:23.600839+00:00 bl-cassoop-p11 kernel: [74795.647343] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.624714+00:00 bl-cassoop-p11 kernel: [74795.668437] BUG: soft lockup - CPU#16 stuck for 22s! [kworker/u97:9:42017]
2014-01-18T13:50:23.624723+00:00 bl-cassoop-p11 kernel: [74795.671173] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.624726+00:00 bl-cassoop-p11 kernel: [74795.671211] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.624727+00:00 bl-cassoop-p11 kernel: [74795.671213] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.624729+00:00 bl-cassoop-p11 kernel: [74795.671216] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:50:23.624731+00:00 bl-cassoop-p11 kernel: [74795.671218] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:50:23.624733+00:00 bl-cassoop-p11 kernel: [74795.671220] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.624734+00:00 bl-cassoop-p11 kernel: [74795.671224] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:50:23.624736+00:00 bl-cassoop-p11 kernel: [74795.671225] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa98720
2014-01-18T13:50:23.624737+00:00 bl-cassoop-p11 kernel: [74795.671227] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.624739+00:00 bl-cassoop-p11 kernel: [74795.671229] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:50:23.624740+00:00 bl-cassoop-p11 kernel: [74795.671231] R10: 0000000000000004 R11: 0000000000000210 R12: ffffffff81254cb7
2014-01-18T13:50:23.624742+00:00 bl-cassoop-p11 kernel: [74795.671232] R13: ffff880075d0b5c8 R14: 0000000000000292 R15: ffff880075d0b5d8
2014-01-18T13:50:23.624743+00:00 bl-cassoop-p11 kernel: [74795.671234] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.624745+00:00 bl-cassoop-p11 kernel: [74795.671236] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.624746+00:00 bl-cassoop-p11 kernel: [74795.671238] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:23.624747+00:00 bl-cassoop-p11 kernel: [74795.671239] Stack:
2014-01-18T13:50:23.624749+00:00 bl-cassoop-p11 kernel: [74795.671240]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:50:23.624750+00:00 bl-cassoop-p11 kernel: [74795.671246]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:50:23.624752+00:00 bl-cassoop-p11 kernel: [74795.671250]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:50:23.624753+00:00 bl-cassoop-p11 kernel: [74795.671255] Call Trace:
2014-01-18T13:50:23.624755+00:00 bl-cassoop-p11 kernel: [74795.671259]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.624756+00:00 bl-cassoop-p11 kernel: [74795.671263]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.624758+00:00 bl-cassoop-p11 kernel: [74795.671266]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:23.624759+00:00 bl-cassoop-p11 kernel: [74795.671269]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:50:23.624761+00:00 bl-cassoop-p11 kernel: [74795.671272]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:50:23.624763+00:00 bl-cassoop-p11 kernel: [74795.671276]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:50:23.624764+00:00 bl-cassoop-p11 kernel: [74795.671279]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:50:23.624766+00:00 bl-cassoop-p11 kernel: [74795.671282]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:50:23.624767+00:00 bl-cassoop-p11 kernel: [74795.671286]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:50:23.624769+00:00 bl-cassoop-p11 kernel: [74795.671289]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:50:23.624771+00:00 bl-cassoop-p11 kernel: [74795.671293]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:50:23.624773+00:00 bl-cassoop-p11 kernel: [74795.671297]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:50:23.624774+00:00 bl-cassoop-p11 kernel: [74795.671300]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:50:23.624776+00:00 bl-cassoop-p11 kernel: [74795.671304]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:50:23.624778+00:00 bl-cassoop-p11 kernel: [74795.671307]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:50:23.624779+00:00 bl-cassoop-p11 kernel: [74795.671311]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:50:23.624781+00:00 bl-cassoop-p11 kernel: [74795.671314]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:50:23.624783+00:00 bl-cassoop-p11 kernel: [74795.671317]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:50:23.624784+00:00 bl-cassoop-p11 kernel: [74795.671321]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:50:23.624786+00:00 bl-cassoop-p11 kernel: [74795.671324]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:50:23.624787+00:00 bl-cassoop-p11 kernel: [74795.671327]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:50:23.624789+00:00 bl-cassoop-p11 kernel: [74795.671330]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:50:23.624790+00:00 bl-cassoop-p11 kernel: [74795.671334]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:50:23.624792+00:00 bl-cassoop-p11 kernel: [74795.671337]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:50:23.624794+00:00 bl-cassoop-p11 kernel: [74795.671341]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:50:23.624795+00:00 bl-cassoop-p11 kernel: [74795.671344]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:50:23.624797+00:00 bl-cassoop-p11 kernel: [74795.671347]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:23.624799+00:00 bl-cassoop-p11 kernel: [74795.671351]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.624800+00:00 bl-cassoop-p11 kernel: [74795.671354]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:23.624802+00:00 bl-cassoop-p11 kernel: [74795.671358]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.624807+00:00 bl-cassoop-p11 kernel: [74795.671359] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.636664+00:00 bl-cassoop-p11 kernel: [74795.680441] BUG: soft lockup - CPU#17 stuck for 22s! [java:42521]
2014-01-18T13:50:23.636673+00:00 bl-cassoop-p11 kernel: [74795.683177] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.636676+00:00 bl-cassoop-p11 kernel: [74795.683200] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.636677+00:00 bl-cassoop-p11 kernel: [74795.683201] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.636679+00:00 bl-cassoop-p11 kernel: [74795.683203] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:50:23.636681+00:00 bl-cassoop-p11 kernel: [74795.683205] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.636683+00:00 bl-cassoop-p11 kernel: [74795.683209] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:50:23.636684+00:00 bl-cassoop-p11 kernel: [74795.683210] RAX: 0000000000000006 RBX: ffffffff81d03da0 RCX: ffff88181fa78748
2014-01-18T13:50:23.636686+00:00 bl-cassoop-p11 kernel: [74795.683212] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.636687+00:00 bl-cassoop-p11 kernel: [74795.683214] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:50:23.636689+00:00 bl-cassoop-p11 kernel: [74795.683215] R10: 0000000000000004 R11: 0000000000017960 R12: ffff880cff722548
2014-01-18T13:50:23.636690+00:00 bl-cassoop-p11 kernel: [74795.683217] R13: 0000000000000000 R14: 0000000000000292 R15: ffff882feccbf938
2014-01-18T13:50:23.636692+00:00 bl-cassoop-p11 kernel: [74795.683219] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.636693+00:00 bl-cassoop-p11 kernel: [74795.683221] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.636695+00:00 bl-cassoop-p11 kernel: [74795.683222] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:50:23.636696+00:00 bl-cassoop-p11 kernel: [74795.683224] Stack:
2014-01-18T13:50:23.636697+00:00 bl-cassoop-p11 kernel: [74795.683225]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:50:23.636699+00:00 bl-cassoop-p11 kernel: [74795.683230]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:50:23.636700+00:00 bl-cassoop-p11 kernel: [74795.683235]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:50:23.636701+00:00 bl-cassoop-p11 kernel: [74795.683239] Call Trace:
2014-01-18T13:50:23.636703+00:00 bl-cassoop-p11 kernel: [74795.683243]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.636705+00:00 bl-cassoop-p11 kernel: [74795.683247]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.636706+00:00 bl-cassoop-p11 kernel: [74795.683250]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:23.636708+00:00 bl-cassoop-p11 kernel: [74795.683253]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:50:23.636710+00:00 bl-cassoop-p11 kernel: [74795.683256]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:50:23.636711+00:00 bl-cassoop-p11 kernel: [74795.683260]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:50:23.636713+00:00 bl-cassoop-p11 kernel: [74795.683265]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:50:23.636714+00:00 bl-cassoop-p11 kernel: [74795.683267]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:50:23.636716+00:00 bl-cassoop-p11 kernel: [74795.683270]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:50:23.636718+00:00 bl-cassoop-p11 kernel: [74795.683273]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:50:23.636719+00:00 bl-cassoop-p11 kernel: [74795.683276]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:50:23.636721+00:00 bl-cassoop-p11 kernel: [74795.683280]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:50:23.636722+00:00 bl-cassoop-p11 kernel: [74795.683282]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:50:23.636724+00:00 bl-cassoop-p11 kernel: [74795.683287]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:50:23.636726+00:00 bl-cassoop-p11 kernel: [74795.683290]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:23.636727+00:00 bl-cassoop-p11 kernel: [74795.683294]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:23.636729+00:00 bl-cassoop-p11 kernel: [74795.683297]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:50:23.636731+00:00 bl-cassoop-p11 kernel: [74795.683300]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:50:23.636732+00:00 bl-cassoop-p11 kernel: [74795.683304]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:50:23.636734+00:00 bl-cassoop-p11 kernel: [74795.683307]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:50:23.636735+00:00 bl-cassoop-p11 kernel: [74795.683310]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:50:23.636737+00:00 bl-cassoop-p11 kernel: [74795.683313]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:23.636738+00:00 bl-cassoop-p11 kernel: [74795.683318]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:23.636741+00:00 bl-cassoop-p11 kernel: [74795.683319] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.648668+00:00 bl-cassoop-p11 kernel: [74795.692446] BUG: soft lockup - CPU#18 stuck for 22s! [kswapd0:283]
2014-01-18T13:50:23.648677+00:00 bl-cassoop-p11 kernel: [74795.695155] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.648680+00:00 bl-cassoop-p11 kernel: [74795.695178] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.648682+00:00 bl-cassoop-p11 kernel: [74795.695179] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.648683+00:00 bl-cassoop-p11 kernel: [74795.695181] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:50:23.648685+00:00 bl-cassoop-p11 kernel: [74795.695183] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:50:23.648687+00:00 bl-cassoop-p11 kernel: [74795.695186] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:50:23.648688+00:00 bl-cassoop-p11 kernel: [74795.695188] RAX: 0000000000000282 RBX: 0000000000000000 RCX: 0000000000000001
2014-01-18T13:50:23.648690+00:00 bl-cassoop-p11 kernel: [74795.695190] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:50:23.648691+00:00 bl-cassoop-p11 kernel: [74795.695191] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:50:23.648693+00:00 bl-cassoop-p11 kernel: [74795.695193] R10: 0000000000000001 R11: 0000000000000400 R12: ffffffff810a2fcc
2014-01-18T13:50:23.648694+00:00 bl-cassoop-p11 kernel: [74795.695195] R13: ffff882ff8361698 R14: ffff8817f2191200 R15: ffff882ff8361648
2014-01-18T13:50:23.648696+00:00 bl-cassoop-p11 kernel: [74795.695197] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.648697+00:00 bl-cassoop-p11 kernel: [74795.695199] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.648699+00:00 bl-cassoop-p11 kernel: [74795.695200] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:23.648700+00:00 bl-cassoop-p11 kernel: [74795.695202] Stack:
2014-01-18T13:50:23.648701+00:00 bl-cassoop-p11 kernel: [74795.695203]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:50:23.648703+00:00 bl-cassoop-p11 kernel: [74795.695209]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:50:23.648704+00:00 bl-cassoop-p11 kernel: [74795.695213]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:50:23.648705+00:00 bl-cassoop-p11 kernel: [74795.695218] Call Trace:
2014-01-18T13:50:23.648707+00:00 bl-cassoop-p11 kernel: [74795.695222]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.648708+00:00 bl-cassoop-p11 kernel: [74795.695225]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:23.648710+00:00 bl-cassoop-p11 kernel: [74795.695229]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.648712+00:00 bl-cassoop-p11 kernel: [74795.695232]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.648713+00:00 bl-cassoop-p11 kernel: [74795.695235]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:23.648715+00:00 bl-cassoop-p11 kernel: [74795.695239]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.648717+00:00 bl-cassoop-p11 kernel: [74795.695242]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.648718+00:00 bl-cassoop-p11 kernel: [74795.695246]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:23.648720+00:00 bl-cassoop-p11 kernel: [74795.695250]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:50:23.648722+00:00 bl-cassoop-p11 kernel: [74795.695253]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:50:23.648723+00:00 bl-cassoop-p11 kernel: [74795.695256]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:50:23.648725+00:00 bl-cassoop-p11 kernel: [74795.695260]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:50:23.648727+00:00 bl-cassoop-p11 kernel: [74795.695264]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:50:23.648728+00:00 bl-cassoop-p11 kernel: [74795.695267]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:50:23.648730+00:00 bl-cassoop-p11 kernel: [74795.695271]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:50:23.648732+00:00 bl-cassoop-p11 kernel: [74795.695274]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:50:23.648733+00:00 bl-cassoop-p11 kernel: [74795.695278]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:23.648735+00:00 bl-cassoop-p11 kernel: [74795.695281]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:23.648736+00:00 bl-cassoop-p11 kernel: [74795.695285]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:23.648738+00:00 bl-cassoop-p11 kernel: [74795.695288]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:23.648740+00:00 bl-cassoop-p11 kernel: [74795.695292]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:23.648741+00:00 bl-cassoop-p11 kernel: [74795.695295]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:50:23.648743+00:00 bl-cassoop-p11 kernel: [74795.695299]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:50:23.648744+00:00 bl-cassoop-p11 kernel: [74795.695302]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:50:23.648746+00:00 bl-cassoop-p11 kernel: [74795.695306]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:50:23.648747+00:00 bl-cassoop-p11 kernel: [74795.695309]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:50:23.648749+00:00 bl-cassoop-p11 kernel: [74795.695312]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:23.648751+00:00 bl-cassoop-p11 kernel: [74795.695316]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.648752+00:00 bl-cassoop-p11 kernel: [74795.695320]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:23.648754+00:00 bl-cassoop-p11 kernel: [74795.695323]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:23.648759+00:00 bl-cassoop-p11 kernel: [74795.695324] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:50:23.672516+00:00 bl-cassoop-p11 kernel: [74795.716452] BUG: soft lockup - CPU#20 stuck for 22s! [jsvc:42019]
2014-01-18T13:50:23.672525+00:00 bl-cassoop-p11 kernel: [74795.719138] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.672528+00:00 bl-cassoop-p11 kernel: [74795.719154] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.672530+00:00 bl-cassoop-p11 kernel: [74795.719155] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.672531+00:00 bl-cassoop-p11 kernel: [74795.719157] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:50:23.672534+00:00 bl-cassoop-p11 kernel: [74795.719158] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:23.672535+00:00 bl-cassoop-p11 kernel: [74795.719161] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:50:23.672537+00:00 bl-cassoop-p11 kernel: [74795.719162] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa387c0
2014-01-18T13:50:23.672538+00:00 bl-cassoop-p11 kernel: [74795.719163] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:23.672540+00:00 bl-cassoop-p11 kernel: [74795.719164] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:50:23.672541+00:00 bl-cassoop-p11 kernel: [74795.719165] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:50:23.672543+00:00 bl-cassoop-p11 kernel: [74795.719166] R13: ffff8801111b3c48 R14: 0000000000000292 R15: ffff8801111b3c78
2014-01-18T13:50:23.672544+00:00 bl-cassoop-p11 kernel: [74795.719168] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.672546+00:00 bl-cassoop-p11 kernel: [74795.719169] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.672547+00:00 bl-cassoop-p11 kernel: [74795.719170] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:50:23.672548+00:00 bl-cassoop-p11 kernel: [74795.719171] Stack:
2014-01-18T13:50:23.672550+00:00 bl-cassoop-p11 kernel: [74795.719172]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:50:23.672551+00:00 bl-cassoop-p11 kernel: [74795.719176]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:50:23.672553+00:00 bl-cassoop-p11 kernel: [74795.719179]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:50:23.672554+00:00 bl-cassoop-p11 kernel: [74795.719182] Call Trace:
2014-01-18T13:50:23.672556+00:00 bl-cassoop-p11 kernel: [74795.719185]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.672558+00:00 bl-cassoop-p11 kernel: [74795.719188]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:23.672559+00:00 bl-cassoop-p11 kernel: [74795.719191]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:50:23.672561+00:00 bl-cassoop-p11 kernel: [74795.719193]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:50:23.672563+00:00 bl-cassoop-p11 kernel: [74795.719195]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:50:23.672564+00:00 bl-cassoop-p11 kernel: [74795.719198]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:50:23.672566+00:00 bl-cassoop-p11 kernel: [74795.719200]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:50:23.672567+00:00 bl-cassoop-p11 kernel: [74795.719203]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:50:23.672569+00:00 bl-cassoop-p11 kernel: [74795.719206]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:23.672572+00:00 bl-cassoop-p11 kernel: [74795.719207] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:23.756641+00:00 bl-cassoop-p11 kernel: [74795.800483] BUG: soft lockup - CPU#27 stuck for 23s! [java:42434]
2014-01-18T13:50:23.756661+00:00 bl-cassoop-p11 kernel: [74795.803183] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.756665+00:00 bl-cassoop-p11 kernel: [74795.803207] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.756667+00:00 bl-cassoop-p11 kernel: [74795.803208] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.756668+00:00 bl-cassoop-p11 kernel: [74795.803210] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:50:23.756671+00:00 bl-cassoop-p11 kernel: [74795.803211] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:50:23.756672+00:00 bl-cassoop-p11 kernel: [74795.803214] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:50:23.756674+00:00 bl-cassoop-p11 kernel: [74795.803215] RAX: 0000000000000030 RBX: ffffffff810a3789 RCX: 0000000000000010
2014-01-18T13:50:23.756675+00:00 bl-cassoop-p11 kernel: [74795.803216] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:50:23.756677+00:00 bl-cassoop-p11 kernel: [74795.803217] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:50:23.756678+00:00 bl-cassoop-p11 kernel: [74795.803218] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:50:23.756680+00:00 bl-cassoop-p11 kernel: [74795.803219] R13: 000000000000000e R14: ffffffff810a108e R15: ffff881887d733b8
2014-01-18T13:50:23.756681+00:00 bl-cassoop-p11 kernel: [74795.803220] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.756682+00:00 bl-cassoop-p11 kernel: [74795.803222] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.756684+00:00 bl-cassoop-p11 kernel: [74795.803223] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:50:23.756685+00:00 bl-cassoop-p11 kernel: [74795.803224] Stack:
2014-01-18T13:50:23.756686+00:00 bl-cassoop-p11 kernel: [74795.803224]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:50:23.756688+00:00 bl-cassoop-p11 kernel: [74795.803228]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:50:23.756690+00:00 bl-cassoop-p11 kernel: [74795.803231]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:50:23.756691+00:00 bl-cassoop-p11 kernel: [74795.803234] Call Trace:
2014-01-18T13:50:23.756692+00:00 bl-cassoop-p11 kernel: [74795.803237]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.756694+00:00 bl-cassoop-p11 kernel: [74795.803239]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:23.756695+00:00 bl-cassoop-p11 kernel: [74795.803242]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.756697+00:00 bl-cassoop-p11 kernel: [74795.803244]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.756699+00:00 bl-cassoop-p11 kernel: [74795.803246]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:23.756700+00:00 bl-cassoop-p11 kernel: [74795.803248]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.756702+00:00 bl-cassoop-p11 kernel: [74795.803251]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:23.756704+00:00 bl-cassoop-p11 kernel: [74795.803254]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:23.756705+00:00 bl-cassoop-p11 kernel: [74795.803256]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:50:23.756707+00:00 bl-cassoop-p11 kernel: [74795.803260]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:50:23.756709+00:00 bl-cassoop-p11 kernel: [74795.803262]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:50:23.756710+00:00 bl-cassoop-p11 kernel: [74795.803264]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:50:23.756712+00:00 bl-cassoop-p11 kernel: [74795.803267]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:50:23.756714+00:00 bl-cassoop-p11 kernel: [74795.803269]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:50:23.756715+00:00 bl-cassoop-p11 kernel: [74795.803272]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:23.756717+00:00 bl-cassoop-p11 kernel: [74795.803274]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:23.756719+00:00 bl-cassoop-p11 kernel: [74795.803277]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:23.756720+00:00 bl-cassoop-p11 kernel: [74795.803280]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:50:23.756722+00:00 bl-cassoop-p11 kernel: [74795.803283]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:50:23.756723+00:00 bl-cassoop-p11 kernel: [74795.803286]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:50:23.756725+00:00 bl-cassoop-p11 kernel: [74795.803289]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:50:23.756727+00:00 bl-cassoop-p11 kernel: [74795.803291]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:50:23.756728+00:00 bl-cassoop-p11 kernel: [74795.803294]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:50:23.756730+00:00 bl-cassoop-p11 kernel: [74795.803297]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:50:23.756732+00:00 bl-cassoop-p11 kernel: [74795.803300]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:50:23.756734+00:00 bl-cassoop-p11 kernel: [74795.803302]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:23.756735+00:00 bl-cassoop-p11 kernel: [74795.803305]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:23.756737+00:00 bl-cassoop-p11 kernel: [74795.803308]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:50:23.756739+00:00 bl-cassoop-p11 kernel: [74795.803310]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:50:23.756740+00:00 bl-cassoop-p11 kernel: [74795.803312]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:50:23.756742+00:00 bl-cassoop-p11 kernel: [74795.803314]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:23.756743+00:00 bl-cassoop-p11 kernel: [74795.803317]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:50:23.756755+00:00 bl-cassoop-p11 kernel: [74795.803319]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:23.756757+00:00 bl-cassoop-p11 kernel: [74795.803321]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:23.756758+00:00 bl-cassoop-p11 kernel: [74795.803327]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:23.756760+00:00 bl-cassoop-p11 kernel: [74795.803330]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:50:23.756762+00:00 bl-cassoop-p11 kernel: [74795.803334]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:50:23.756763+00:00 bl-cassoop-p11 kernel: [74795.803336]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:23.756765+00:00 bl-cassoop-p11 kernel: [74795.803338]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:50:23.756767+00:00 bl-cassoop-p11 kernel: [74795.803340]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:23.756770+00:00 bl-cassoop-p11 kernel: [74795.803342]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:23.756773+00:00 bl-cassoop-p11 kernel: [74795.803343] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:50:23.792597+00:00 bl-cassoop-p11 kernel: [74795.836494] BUG: soft lockup - CPU#30 stuck for 23s! [java:36244]
2014-01-18T13:50:23.792606+00:00 bl-cassoop-p11 kernel: [74795.839153] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:23.792609+00:00 bl-cassoop-p11 kernel: [74795.839175] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:23.792610+00:00 bl-cassoop-p11 kernel: [74795.839176] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:23.792612+00:00 bl-cassoop-p11 kernel: [74795.839178] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:50:23.792614+00:00 bl-cassoop-p11 kernel: [74795.839180] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:50:23.792616+00:00 bl-cassoop-p11 kernel: [74795.839183] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:50:23.792617+00:00 bl-cassoop-p11 kernel: [74795.839185] RAX: 0000000000000286 RBX: ffff88187fffcb08 RCX: 0000000000000001
2014-01-18T13:50:23.792619+00:00 bl-cassoop-p11 kernel: [74795.839186] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:50:23.792620+00:00 bl-cassoop-p11 kernel: [74795.839188] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:50:23.792622+00:00 bl-cassoop-p11 kernel: [74795.839189] R10: 00000000000035de R11: 0000000000000001 R12: 0000000000000000
2014-01-18T13:50:23.792623+00:00 bl-cassoop-p11 kernel: [74795.839191] R13: ffff88187fffcb00 R14: ffff88187fffcb08 R15: ffffffff00000000
2014-01-18T13:50:23.792625+00:00 bl-cassoop-p11 kernel: [74795.839193] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:50:23.792626+00:00 bl-cassoop-p11 kernel: [74795.839195] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:23.792627+00:00 bl-cassoop-p11 kernel: [74795.839197] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:23.792628+00:00 bl-cassoop-p11 kernel: [74795.839198] Stack:
2014-01-18T13:50:23.792630+00:00 bl-cassoop-p11 kernel: [74795.839199]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:50:23.792632+00:00 bl-cassoop-p11 kernel: [74795.839205]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:50:23.792633+00:00 bl-cassoop-p11 kernel: [74795.839209]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:50:23.792634+00:00 bl-cassoop-p11 kernel: [74795.839214] Call Trace:
2014-01-18T13:50:23.792636+00:00 bl-cassoop-p11 kernel: [74795.839217]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.792637+00:00 bl-cassoop-p11 kernel: [74795.839220]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:23.792639+00:00 bl-cassoop-p11 kernel: [74795.839224]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.792641+00:00 bl-cassoop-p11 kernel: [74795.839227]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:23.792642+00:00 bl-cassoop-p11 kernel: [74795.839230]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:23.792644+00:00 bl-cassoop-p11 kernel: [74795.839233]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:50:23.792646+00:00 bl-cassoop-p11 kernel: [74795.839237]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:23.792647+00:00 bl-cassoop-p11 kernel: [74795.839240]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:23.792649+00:00 bl-cassoop-p11 kernel: [74795.839244]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:50:23.792650+00:00 bl-cassoop-p11 kernel: [74795.839247]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:50:23.792652+00:00 bl-cassoop-p11 kernel: [74795.839250]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:50:23.792653+00:00 bl-cassoop-p11 kernel: [74795.839253]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:50:23.792655+00:00 bl-cassoop-p11 kernel: [74795.839256]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:50:23.792657+00:00 bl-cassoop-p11 kernel: [74795.839259]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:50:23.792658+00:00 bl-cassoop-p11 kernel: [74795.839262]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:50:23.792660+00:00 bl-cassoop-p11 kernel: [74795.839265]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:50:23.792661+00:00 bl-cassoop-p11 kernel: [74795.839269]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:50:23.792663+00:00 bl-cassoop-p11 kernel: [74795.839272]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:50:23.792665+00:00 bl-cassoop-p11 kernel: [74795.839275]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:50:23.792666+00:00 bl-cassoop-p11 kernel: [74795.839278]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:50:23.792668+00:00 bl-cassoop-p11 kernel: [74795.839282]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:50:23.792670+00:00 bl-cassoop-p11 kernel: [74795.839285]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:50:23.792671+00:00 bl-cassoop-p11 kernel: [74795.839289]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:50:23.792673+00:00 bl-cassoop-p11 kernel: [74795.839292]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:50:23.792674+00:00 bl-cassoop-p11 kernel: [74795.839295]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:50:23.792676+00:00 bl-cassoop-p11 kernel: [74795.839299]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:50:23.792677+00:00 bl-cassoop-p11 kernel: [74795.839302]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:50:23.792679+00:00 bl-cassoop-p11 kernel: [74795.839306]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:50:23.792682+00:00 bl-cassoop-p11 kernel: [74795.839307] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:50:51.443267+00:00 bl-cassoop-p11 kernel: [74823.494577] BUG: soft lockup - CPU#1 stuck for 22s! [collectd:5830]
2014-01-18T13:50:51.443275+00:00 bl-cassoop-p11 kernel: [74823.494580] BUG: soft lockup - CPU#0 stuck for 22s! [java:42506]
2014-01-18T13:50:51.443279+00:00 bl-cassoop-p11 kernel: [74823.494596] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.443282+00:00 bl-cassoop-p11 kernel: [74823.494598] CPU: 0 PID: 42506 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.443284+00:00 bl-cassoop-p11 kernel: [74823.494599] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.443286+00:00 bl-cassoop-p11 kernel: [74823.494600] task: ffff88002c269770 ti: ffff880045cce000 task.ti: ffff880045cce000
2014-01-18T13:50:51.443288+00:00 bl-cassoop-p11 kernel: [74823.494603] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:51.443289+00:00 bl-cassoop-p11 kernel: [74823.494604] RSP: 0018:ffff880045ccfcd8  EFLAGS: 00000202
2014-01-18T13:50:51.443291+00:00 bl-cassoop-p11 kernel: [74823.494605] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa97868
2014-01-18T13:50:51.443293+00:00 bl-cassoop-p11 kernel: [74823.494606] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.443294+00:00 bl-cassoop-p11 kernel: [74823.494607] RBP: ffff880045ccfd38 R08: ffff88181fa15090 R09: 0000000000000000
2014-01-18T13:50:51.443296+00:00 bl-cassoop-p11 kernel: [74823.494608] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:50:51.443297+00:00 bl-cassoop-p11 kernel: [74823.494608] R13: ffff880045ccfc48 R14: 0000000000000292 R15: ffff880045ccfc78
2014-01-18T13:50:51.443299+00:00 bl-cassoop-p11 kernel: [74823.494610] FS:  00007f58f4369700(0000) GS:ffff88181fa00000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.443300+00:00 bl-cassoop-p11 kernel: [74823.494611] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.443302+00:00 bl-cassoop-p11 kernel: [74823.494612] CR2: 00007f5918b9e1a8 CR3: 0000002ff6d16000 CR4: 00000000000407f0
2014-01-18T13:50:51.443303+00:00 bl-cassoop-p11 kernel: [74823.494612] Stack:
2014-01-18T13:50:51.443304+00:00 bl-cassoop-p11 kernel: [74823.494618]  ffff88181faf5050 ffff88181faf5040 ffff88181faf7868 ffff882f0000000e
2014-01-18T13:50:51.443306+00:00 bl-cassoop-p11 kernel: [74823.494621]  01ff880045ccfd08 ffff88181fa150a8 ffff880045ccfd28 ffff882ff7e6e130
2014-01-18T13:50:51.443307+00:00 bl-cassoop-p11 kernel: [74823.494624]  ffff882ff7e6de80 00007f58f426c000 0000000000000000 ffff882ff7e6de80
2014-01-18T13:50:51.443309+00:00 bl-cassoop-p11 kernel: [74823.494625] Call Trace:
2014-01-18T13:50:51.443310+00:00 bl-cassoop-p11 kernel: [74823.494628]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.443312+00:00 bl-cassoop-p11 kernel: [74823.494631]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:51.443314+00:00 bl-cassoop-p11 kernel: [74823.494634]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:50:51.443315+00:00 bl-cassoop-p11 kernel: [74823.494637]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:50:51.443317+00:00 bl-cassoop-p11 kernel: [74823.494639]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:50:51.443318+00:00 bl-cassoop-p11 kernel: [74823.494641]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:50:51.443320+00:00 bl-cassoop-p11 kernel: [74823.494644]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:50:51.443321+00:00 bl-cassoop-p11 kernel: [74823.494646]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:50:51.443323+00:00 bl-cassoop-p11 kernel: [74823.494649]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:51.443326+00:00 bl-cassoop-p11 kernel: [74823.494668] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:51.443330+00:00 bl-cassoop-p11 kernel: [74823.499940] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.443332+00:00 bl-cassoop-p11 kernel: [74823.499965] CPU: 1 PID: 5830 Comm: collectd Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.443334+00:00 bl-cassoop-p11 kernel: [74823.499967] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.443335+00:00 bl-cassoop-p11 kernel: [74823.499969] task: ffff882ff0912ee0 ti: ffff882ffa6f6000 task.ti: ffff882ffa6f6000
2014-01-18T13:50:51.443337+00:00 bl-cassoop-p11 kernel: [74823.499970] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:50:51.443339+00:00 bl-cassoop-p11 kernel: [74823.499976] RSP: 0018:ffff882ffa6f7d08  EFLAGS: 00000202
2014-01-18T13:50:51.443340+00:00 bl-cassoop-p11 kernel: [74823.499977] RAX: 0000000000000286 RBX: 0000000000000010 RCX: 0000000000000001
2014-01-18T13:50:51.443342+00:00 bl-cassoop-p11 kernel: [74823.499979] RDX: ffff88181fa38680 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:50:51.443343+00:00 bl-cassoop-p11 kernel: [74823.499980] RBP: ffff882ffa6f7d48 R08: 0000000000000000 R09: 000000000000730e
2014-01-18T13:50:51.443345+00:00 bl-cassoop-p11 kernel: [74823.499982] R10: 0000000000003708 R11: 0000000000000293 R12: 0000000000000000
2014-01-18T13:50:51.443346+00:00 bl-cassoop-p11 kernel: [74823.499984] R13: 00007f2a9217e000 R14: ffff88307fff9d80 R15: ffff882ffa6f7f24
2014-01-18T13:50:51.443348+00:00 bl-cassoop-p11 kernel: [74823.499986] FS:  00007f2a8a89f700(0000) GS:ffff88301f200000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.443367+00:00 bl-cassoop-p11 kernel: [74823.499987] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.443368+00:00 bl-cassoop-p11 kernel: [74823.499989] CR2: 00007f2a92182000 CR3: 0000002ff0b0e000 CR4: 00000000000407e0
2014-01-18T13:50:51.443370+00:00 bl-cassoop-p11 kernel: [74823.499990] Stack:
2014-01-18T13:50:51.443371+00:00 bl-cassoop-p11 kernel: [74823.499992]  ffff882ff7817000 ffff88181fa386a8 00000007f2a9217e 0000000000000002
2014-01-18T13:50:51.443373+00:00 bl-cassoop-p11 kernel: [74823.499998]  ffffffff8139a810 0000000000000001 ffffffff81d03da0 00007f2a92182000
2014-01-18T13:50:51.443374+00:00 bl-cassoop-p11 kernel: [74823.500003]  ffff882ffa6f7db8 ffffffff810cb5a5 ffff882ff0912ee0 0000000000000001
2014-01-18T13:50:51.443375+00:00 bl-cassoop-p11 kernel: [74823.500007] Call Trace:
2014-01-18T13:50:51.443377+00:00 bl-cassoop-p11 kernel: [74823.500013]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:50:51.443379+00:00 bl-cassoop-p11 kernel: [74823.500017]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:51.443380+00:00 bl-cassoop-p11 kernel: [74823.500020]  [<ffffffff8139a810>] ? ucs2_strncmp+0x70/0x70
2014-01-18T13:50:51.443382+00:00 bl-cassoop-p11 kernel: [74823.500024]  [<ffffffff8139ab4e>] rdmsr_on_cpu+0x4e/0x70
2014-01-18T13:50:51.443384+00:00 bl-cassoop-p11 kernel: [74823.500029]  [<ffffffffa001b62b>] show_temp+0xab/0xf0 [coretemp]
2014-01-18T13:50:51.443385+00:00 bl-cassoop-p11 kernel: [74823.500033]  [<ffffffff81490f20>] dev_attr_show+0x20/0x60
2014-01-18T13:50:51.443387+00:00 bl-cassoop-p11 kernel: [74823.500038]  [<ffffffff8114d37e>] ? __get_free_pages+0xe/0x40
2014-01-18T13:50:51.443389+00:00 bl-cassoop-p11 kernel: [74823.500043]  [<ffffffff81227e86>] fill_read_buffer.isra.9+0x66/0xf0
2014-01-18T13:50:51.443393+00:00 bl-cassoop-p11 kernel: [74823.500046]  [<ffffffff81227fc0>] sysfs_read_file+0xb0/0xf0
2014-01-18T13:50:51.443395+00:00 bl-cassoop-p11 kernel: [74823.500050]  [<ffffffff811b42e4>] vfs_read+0xb4/0x180
2014-01-18T13:50:51.443396+00:00 bl-cassoop-p11 kernel: [74823.500053]  [<ffffffff811b4562>] SyS_read+0x52/0xa0
2014-01-18T13:50:51.443398+00:00 bl-cassoop-p11 kernel: [74823.500058]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:51.443401+00:00 bl-cassoop-p11 kernel: [74823.500059] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:50:51.552377+00:00 bl-cassoop-p11 kernel: [74823.606619] BUG: soft lockup - CPU#10 stuck for 22s! [java:42435]
2014-01-18T13:50:51.552387+00:00 bl-cassoop-p11 kernel: [74823.609110] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.552390+00:00 bl-cassoop-p11 kernel: [74823.609137] CPU: 10 PID: 42435 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.552392+00:00 bl-cassoop-p11 kernel: [74823.609138] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.552394+00:00 bl-cassoop-p11 kernel: [74823.609139] task: ffff8829030e2ee0 ti: ffff8819e9a80000 task.ti: ffff8819e9a80000
2014-01-18T13:50:51.552396+00:00 bl-cassoop-p11 kernel: [74823.609141] RIP: 0010:[<ffffffff817310e6>]  [<ffffffff817310e6>] oom_kill_process.part.6+0x12c/0x2cf
2014-01-18T13:50:51.552397+00:00 bl-cassoop-p11 kernel: [74823.609144] RSP: 0000:ffff8819e9a81a28  EFLAGS: 00000246
2014-01-18T13:50:51.552399+00:00 bl-cassoop-p11 kernel: [74823.609145] RAX: ffff881894a69b00 RBX: 00000000000000d0 RCX: 0000000000000006
2014-01-18T13:50:51.552400+00:00 bl-cassoop-p11 kernel: [74823.609147] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffffffff81c06040
2014-01-18T13:50:51.552402+00:00 bl-cassoop-p11 kernel: [74823.609148] RBP: ffff8819e9a81a88 R08: 0000000000000000 R09: 0000000000000001
2014-01-18T13:50:51.552403+00:00 bl-cassoop-p11 kernel: [74823.609149] R10: 0000000000000e32 R11: 0000000000005000 R12: ffff882ff044e3f0
2014-01-18T13:50:51.552405+00:00 bl-cassoop-p11 kernel: [74823.609150] R13: 0000000000000005 R14: 0000000000000246 R15: 0000000028e74000
2014-01-18T13:50:51.552407+00:00 bl-cassoop-p11 kernel: [74823.609151] FS:  00007f636ff99700(0000) GS:ffff88181faa0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.552408+00:00 bl-cassoop-p11 kernel: [74823.609152] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.552410+00:00 bl-cassoop-p11 kernel: [74823.609153] CR2: 00000000ace4d000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:50:51.552411+00:00 bl-cassoop-p11 kernel: [74823.609154] Stack:
2014-01-18T13:50:51.552412+00:00 bl-cassoop-p11 kernel: [74823.609155]  ffff8819e9a81a38 ffff882ff044ddc0 000000000114eccd ffffc90028e74000
2014-01-18T13:50:51.552414+00:00 bl-cassoop-p11 kernel: [74823.609159]  ffff8819e9a81a88 ffff881894a69b00 ffff882ff5c55dc0 000000000114eccd
2014-01-18T13:50:51.552415+00:00 bl-cassoop-p11 kernel: [74823.609162]  0000000000000000 ffffc90028e74000 ffff882ff0b86a00 ffff882ff044ddc0
2014-01-18T13:50:51.552416+00:00 bl-cassoop-p11 kernel: [74823.609165] Call Trace:
2014-01-18T13:50:51.552418+00:00 bl-cassoop-p11 kernel: [74823.609168]  [<ffffffff8114c6dd>] oom_kill_process+0x4d/0x50
2014-01-18T13:50:51.552419+00:00 bl-cassoop-p11 kernel: [74823.609170]  [<ffffffff81733a5f>] mem_cgroup_out_of_memory+0x22c/0x255
2014-01-18T13:50:51.552421+00:00 bl-cassoop-p11 kernel: [74823.609172]  [<ffffffff81733bc7>] mem_cgroup_handle_oom+0x13f/0x24a
2014-01-18T13:50:51.552423+00:00 bl-cassoop-p11 kernel: [74823.609175]  [<ffffffff811a68ee>] ? mem_cgroup_margin+0x4e/0xc0
2014-01-18T13:50:51.552425+00:00 bl-cassoop-p11 kernel: [74823.609177]  [<ffffffff811ab8d0>] ? mem_cgroup_charge_common+0xa0/0xa0
2014-01-18T13:50:51.552426+00:00 bl-cassoop-p11 kernel: [74823.609180]  [<ffffffff811aae21>] mem_cgroup_do_charge+0x151/0x180
2014-01-18T13:50:51.552428+00:00 bl-cassoop-p11 kernel: [74823.609183]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:51.552430+00:00 bl-cassoop-p11 kernel: [74823.609185]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:51.552431+00:00 bl-cassoop-p11 kernel: [74823.609188]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:50:51.552433+00:00 bl-cassoop-p11 kernel: [74823.609190]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:50:51.552435+00:00 bl-cassoop-p11 kernel: [74823.609192]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:50:51.552436+00:00 bl-cassoop-p11 kernel: [74823.609195]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:51.552438+00:00 bl-cassoop-p11 kernel: [74823.609197]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:50:51.552440+00:00 bl-cassoop-p11 kernel: [74823.609200]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:51.552441+00:00 bl-cassoop-p11 kernel: [74823.609202]  [<ffffffff8108d7ec>] ? down_read_trylock+0xc/0x30
2014-01-18T13:50:51.552443+00:00 bl-cassoop-p11 kernel: [74823.609204]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:51.552445+00:00 bl-cassoop-p11 kernel: [74823.609207]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:51.552446+00:00 bl-cassoop-p11 kernel: [74823.609209]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:50:51.552448+00:00 bl-cassoop-p11 kernel: [74823.609211]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:50:51.552450+00:00 bl-cassoop-p11 kernel: [74823.609213]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:51.552451+00:00 bl-cassoop-p11 kernel: [74823.609216]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:50:51.552453+00:00 bl-cassoop-p11 kernel: [74823.609218]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:51.552454+00:00 bl-cassoop-p11 kernel: [74823.609220]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:51.552457+00:00 bl-cassoop-p11 kernel: [74823.609221] Code: 89 c5 48 8b 9b 10 03 00 00 48 81 eb 10 03 00 00 48 8d 83 10 03 00 00 4c 39 e0 75 a1 49 8b 86 90 03 00 00 48 89 45 c8 4c 8b 75 c8 <49> 81 ee 90 03 00 00 4d 39 f7 0f 85 6b ff ff ff f0 ff 05 43 4f
2014-01-18T13:50:51.576178+00:00 bl-cassoop-p11 kernel: [74823.630626] BUG: soft lockup - CPU#12 stuck for 22s! [jsvc:7242]
2014-01-18T13:50:51.576186+00:00 bl-cassoop-p11 kernel: [74823.632951] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.576189+00:00 bl-cassoop-p11 kernel: [74823.632967] CPU: 12 PID: 7242 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.576191+00:00 bl-cassoop-p11 kernel: [74823.632968] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.576193+00:00 bl-cassoop-p11 kernel: [74823.632969] task: ffff881736d3c650 ti: ffff8817f8e44000 task.ti: ffff8817f8e44000
2014-01-18T13:50:51.576195+00:00 bl-cassoop-p11 kernel: [74823.632971] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:51.576197+00:00 bl-cassoop-p11 kernel: [74823.632974] RSP: 0000:ffff8817f8e45998  EFLAGS: 00000202
2014-01-18T13:50:51.576198+00:00 bl-cassoop-p11 kernel: [74823.632975] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa38680
2014-01-18T13:50:51.576200+00:00 bl-cassoop-p11 kernel: [74823.632976] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.576201+00:00 bl-cassoop-p11 kernel: [74823.632977] RBP: ffff8817f8e459f8 R08: ffff88181fad5090 R09: 0000000000000000
2014-01-18T13:50:51.576203+00:00 bl-cassoop-p11 kernel: [74823.632978] R10: 0000000000000004 R11: 00000000000179c8 R12: ffff8817f37ca000
2014-01-18T13:50:51.576204+00:00 bl-cassoop-p11 kernel: [74823.632979] R13: 0000000000000001 R14: 0000000000000292 R15: ffff8817f8e45938
2014-01-18T13:50:51.576206+00:00 bl-cassoop-p11 kernel: [74823.632981] FS:  00007ee5ad4f8700(0000) GS:ffff88181fac0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.576207+00:00 bl-cassoop-p11 kernel: [74823.632982] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.576209+00:00 bl-cassoop-p11 kernel: [74823.632983] CR2: 00000006dc028000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:50:51.576210+00:00 bl-cassoop-p11 kernel: [74823.632984] Stack:
2014-01-18T13:50:51.576211+00:00 bl-cassoop-p11 kernel: [74823.632985]  ffff88181fb55050 ffff88181fb55040 ffff88181fb58680 0000000000000014
2014-01-18T13:50:51.576213+00:00 bl-cassoop-p11 kernel: [74823.632988]  0100000000000000 ffff88181fad50a8 ffffffff00000000 ffff882ff7e68380
2014-01-18T13:50:51.576214+00:00 bl-cassoop-p11 kernel: [74823.632991]  ffff882ff7e68630 00000006dc028000 ffff882ff7e68380 ffffea0096b20940
2014-01-18T13:50:51.576215+00:00 bl-cassoop-p11 kernel: [74823.632994] Call Trace:
2014-01-18T13:50:51.576217+00:00 bl-cassoop-p11 kernel: [74823.632998]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.576219+00:00 bl-cassoop-p11 kernel: [74823.633000]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.576220+00:00 bl-cassoop-p11 kernel: [74823.633002]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:51.576222+00:00 bl-cassoop-p11 kernel: [74823.633005]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:50:51.576224+00:00 bl-cassoop-p11 kernel: [74823.633007]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:50:51.576225+00:00 bl-cassoop-p11 kernel: [74823.633010]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:50:51.576227+00:00 bl-cassoop-p11 kernel: [74823.633012]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:50:51.576228+00:00 bl-cassoop-p11 kernel: [74823.633014]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:50:51.576230+00:00 bl-cassoop-p11 kernel: [74823.633016]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:50:51.576232+00:00 bl-cassoop-p11 kernel: [74823.633018]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:50:51.576233+00:00 bl-cassoop-p11 kernel: [74823.633020]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:50:51.576235+00:00 bl-cassoop-p11 kernel: [74823.633022]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:50:51.576237+00:00 bl-cassoop-p11 kernel: [74823.633024]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:50:51.576238+00:00 bl-cassoop-p11 kernel: [74823.633026]  [<ffffffff81620a22>] ? sock_aio_read.part.10+0x82/0xa0
2014-01-18T13:50:51.576240+00:00 bl-cassoop-p11 kernel: [74823.633028]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:51.576242+00:00 bl-cassoop-p11 kernel: [74823.633031]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:51.576243+00:00 bl-cassoop-p11 kernel: [74823.633033]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:51.576245+00:00 bl-cassoop-p11 kernel: [74823.633036]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:51.576247+00:00 bl-cassoop-p11 kernel: [74823.633037] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:51.588042+00:00 bl-cassoop-p11 kernel: [74823.642622] BUG: soft lockup - CPU#13 stuck for 22s! [kswapd1:284]
2014-01-18T13:50:51.588050+00:00 bl-cassoop-p11 kernel: [74823.644798] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.588053+00:00 bl-cassoop-p11 kernel: [74823.644813] CPU: 13 PID: 284 Comm: kswapd1 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.588055+00:00 bl-cassoop-p11 kernel: [74823.644815] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.588056+00:00 bl-cassoop-p11 kernel: [74823.644816] task: ffff882ff8368000 ti: ffff882ff8362000 task.ti: ffff882ff8362000
2014-01-18T13:50:51.588059+00:00 bl-cassoop-p11 kernel: [74823.644817] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:50:51.588060+00:00 bl-cassoop-p11 kernel: [74823.644820] RSP: 0018:ffff882ff83637a8  EFLAGS: 00000202
2014-01-18T13:50:51.588061+00:00 bl-cassoop-p11 kernel: [74823.644821] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa386a8
2014-01-18T13:50:51.588063+00:00 bl-cassoop-p11 kernel: [74823.644822] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.588064+00:00 bl-cassoop-p11 kernel: [74823.644823] RBP: ffff882ff8363808 R08: ffff88301f2d5090 R09: 0000000000000000
2014-01-18T13:50:51.588066+00:00 bl-cassoop-p11 kernel: [74823.644824] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000009
2014-01-18T13:50:51.588067+00:00 bl-cassoop-p11 kernel: [74823.644825] R13: ffffffff8105ae90 R14: 0000000000000292 R15: ffff882ff8363748
2014-01-18T13:50:51.588069+00:00 bl-cassoop-p11 kernel: [74823.644827] FS:  0000000000000000(0000) GS:ffff88301f2c0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.588070+00:00 bl-cassoop-p11 kernel: [74823.644828] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.588072+00:00 bl-cassoop-p11 kernel: [74823.644829] CR2: 00000000ae4f8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:51.588073+00:00 bl-cassoop-p11 kernel: [74823.644830] Stack:
2014-01-18T13:50:51.588074+00:00 bl-cassoop-p11 kernel: [74823.644830]  ffff88301f3d5050 ffff88301f3d5040 ffff88301f3d86a8 000000000000001d
2014-01-18T13:50:51.588076+00:00 bl-cassoop-p11 kernel: [74823.644834]  01ff881a3fa58b60 ffff88301f2d50a8 ffff88181f442800 ffff882ff7e68380
2014-01-18T13:50:51.588077+00:00 bl-cassoop-p11 kernel: [74823.644837]  ffff882ff7e68630 00007ee8c94c1000 ffff882ff83639c0 ffff882ff8363944
2014-01-18T13:50:51.588078+00:00 bl-cassoop-p11 kernel: [74823.644840] Call Trace:
2014-01-18T13:50:51.588080+00:00 bl-cassoop-p11 kernel: [74823.644844]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.588082+00:00 bl-cassoop-p11 kernel: [74823.644847]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.588083+00:00 bl-cassoop-p11 kernel: [74823.644849]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:51.588085+00:00 bl-cassoop-p11 kernel: [74823.644852]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:50:51.588087+00:00 bl-cassoop-p11 kernel: [74823.644854]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:50:51.588088+00:00 bl-cassoop-p11 kernel: [74823.644856]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:50:51.588090+00:00 bl-cassoop-p11 kernel: [74823.644859]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:50:51.588092+00:00 bl-cassoop-p11 kernel: [74823.644861]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:50:51.588094+00:00 bl-cassoop-p11 kernel: [74823.644864]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:51.588095+00:00 bl-cassoop-p11 kernel: [74823.644866]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:51.588097+00:00 bl-cassoop-p11 kernel: [74823.644870]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:51.588098+00:00 bl-cassoop-p11 kernel: [74823.644872]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:51.588100+00:00 bl-cassoop-p11 kernel: [74823.644875]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:51.588102+00:00 bl-cassoop-p11 kernel: [74823.644877]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:50:51.588103+00:00 bl-cassoop-p11 kernel: [74823.644879]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:50:51.588105+00:00 bl-cassoop-p11 kernel: [74823.644883]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:50:51.588107+00:00 bl-cassoop-p11 kernel: [74823.644886]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:50:51.588108+00:00 bl-cassoop-p11 kernel: [74823.644888]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:50:51.588110+00:00 bl-cassoop-p11 kernel: [74823.644891]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:51.588111+00:00 bl-cassoop-p11 kernel: [74823.644893]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.588113+00:00 bl-cassoop-p11 kernel: [74823.644896]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:51.588114+00:00 bl-cassoop-p11 kernel: [74823.644898]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.588117+00:00 bl-cassoop-p11 kernel: [74823.644899] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:50:51.600053+00:00 bl-cassoop-p11 kernel: [74823.654636] BUG: soft lockup - CPU#14 stuck for 22s! [java:42064]
2014-01-18T13:50:51.600062+00:00 bl-cassoop-p11 kernel: [74823.656674] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.600065+00:00 bl-cassoop-p11 kernel: [74823.656696] CPU: 14 PID: 42064 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.600067+00:00 bl-cassoop-p11 kernel: [74823.656698] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.600069+00:00 bl-cassoop-p11 kernel: [74823.656700] task: ffff8817f92c9770 ti: ffff88195e364000 task.ti: ffff88195e364000
2014-01-18T13:50:51.600071+00:00 bl-cassoop-p11 kernel: [74823.656701] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:50:51.600072+00:00 bl-cassoop-p11 kernel: [74823.656705] RSP: 0018:ffff88195e365108  EFLAGS: 00000202
2014-01-18T13:50:51.600074+00:00 bl-cassoop-p11 kernel: [74823.656706] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa986d0
2014-01-18T13:50:51.600075+00:00 bl-cassoop-p11 kernel: [74823.656708] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.600077+00:00 bl-cassoop-p11 kernel: [74823.656710] RBP: ffff88195e365168 R08: ffff88181faf5088 R09: 0000000000000000
2014-01-18T13:50:51.600078+00:00 bl-cassoop-p11 kernel: [74823.656711] R10: 0000000000000004 R11: 0000000000000058 R12: ffffffff81049a3e
2014-01-18T13:50:51.600080+00:00 bl-cassoop-p11 kernel: [74823.656713] R13: ffff88195e365078 R14: 0000000000000292 R15: ffff88195e3650a8
2014-01-18T13:50:51.600081+00:00 bl-cassoop-p11 kernel: [74823.656715] FS:  00007f86459f1700(0000) GS:ffff88181fae0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.600083+00:00 bl-cassoop-p11 kernel: [74823.656717] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.600084+00:00 bl-cassoop-p11 kernel: [74823.656718] CR2: 00000000adf99000 CR3: 0000002e0791b000 CR4: 00000000000407e0
2014-01-18T13:50:51.600085+00:00 bl-cassoop-p11 kernel: [74823.656720] Stack:
2014-01-18T13:50:51.600087+00:00 bl-cassoop-p11 kernel: [74823.656721]  ffff88181fa95050 ffff88181fa95040 ffff88181fa986d0 ffff881900000008
2014-01-18T13:50:51.600088+00:00 bl-cassoop-p11 kernel: [74823.656726]  01ff88195e365158 ffff88181faf50a8 ffff88195e365158 ffff882ff7e6de80
2014-01-18T13:50:51.600090+00:00 bl-cassoop-p11 kernel: [74823.656730]  ffff882ff7e6e130 00007f58fd807000 ffff882ff7e6de80 ffffea0099536900
2014-01-18T13:50:51.600091+00:00 bl-cassoop-p11 kernel: [74823.656735] Call Trace:
2014-01-18T13:50:51.600092+00:00 bl-cassoop-p11 kernel: [74823.656739]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.600094+00:00 bl-cassoop-p11 kernel: [74823.656742]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.600096+00:00 bl-cassoop-p11 kernel: [74823.656746]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:51.600097+00:00 bl-cassoop-p11 kernel: [74823.656749]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:50:51.600099+00:00 bl-cassoop-p11 kernel: [74823.656753]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:50:51.600100+00:00 bl-cassoop-p11 kernel: [74823.656756]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:50:51.600102+00:00 bl-cassoop-p11 kernel: [74823.656759]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:50:51.600104+00:00 bl-cassoop-p11 kernel: [74823.656763]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:50:51.600105+00:00 bl-cassoop-p11 kernel: [74823.656767]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:51.600107+00:00 bl-cassoop-p11 kernel: [74823.656770]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:51.600108+00:00 bl-cassoop-p11 kernel: [74823.656774]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:51.600110+00:00 bl-cassoop-p11 kernel: [74823.656777]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:50:51.600112+00:00 bl-cassoop-p11 kernel: [74823.656780]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:50:51.600113+00:00 bl-cassoop-p11 kernel: [74823.656784]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:50:51.600115+00:00 bl-cassoop-p11 kernel: [74823.656787]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:50:51.600117+00:00 bl-cassoop-p11 kernel: [74823.656791]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:50:51.600118+00:00 bl-cassoop-p11 kernel: [74823.656795]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:50:51.600120+00:00 bl-cassoop-p11 kernel: [74823.656798]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:50:51.600121+00:00 bl-cassoop-p11 kernel: [74823.656802]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:51.600123+00:00 bl-cassoop-p11 kernel: [74823.656806]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:51.600125+00:00 bl-cassoop-p11 kernel: [74823.656809]  [<ffffffff811ac58d>] mem_cgroup_cache_charge+0x7d/0x90
2014-01-18T13:50:51.600127+00:00 bl-cassoop-p11 kernel: [74823.656813]  [<ffffffff81149796>] add_to_page_cache_locked+0x46/0x90
2014-01-18T13:50:51.600128+00:00 bl-cassoop-p11 kernel: [74823.656816]  [<ffffffff81149801>] add_to_page_cache_lru+0x21/0x50
2014-01-18T13:50:51.600130+00:00 bl-cassoop-p11 kernel: [74823.656819]  [<ffffffff811f216a>] mpage_readpages+0xba/0x130
2014-01-18T13:50:51.600132+00:00 bl-cassoop-p11 kernel: [74823.656823]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:50:51.600133+00:00 bl-cassoop-p11 kernel: [74823.656826]  [<ffffffff8134b3a7>] ? __blk_run_queue+0x37/0x50
2014-01-18T13:50:51.600135+00:00 bl-cassoop-p11 kernel: [74823.656829]  [<ffffffff81254730>] ? ext4_get_block_write+0x20/0x20
2014-01-18T13:50:51.600136+00:00 bl-cassoop-p11 kernel: [74823.656833]  [<ffffffff8124fc95>] ext4_readpages+0x45/0x60
2014-01-18T13:50:51.600138+00:00 bl-cassoop-p11 kernel: [74823.656836]  [<ffffffff81154b88>] read_pages+0x48/0x100
2014-01-18T13:50:51.600140+00:00 bl-cassoop-p11 kernel: [74823.656839]  [<ffffffff81154d9b>] __do_page_cache_readahead+0x15b/0x170
2014-01-18T13:50:51.600141+00:00 bl-cassoop-p11 kernel: [74823.656842]  [<ffffffff81155111>] ra_submit+0x21/0x30
2014-01-18T13:50:51.600143+00:00 bl-cassoop-p11 kernel: [74823.656845]  [<ffffffff81148f32>] do_sync_mmap_readahead.isra.28+0x92/0xc0
2014-01-18T13:50:51.600160+00:00 bl-cassoop-p11 kernel: [74823.656848]  [<ffffffff8114b473>] filemap_fault+0x353/0x430
2014-01-18T13:50:51.600162+00:00 bl-cassoop-p11 kernel: [74823.656852]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:51.600163+00:00 bl-cassoop-p11 kernel: [74823.656856]  [<ffffffff8116ed32>] __do_fault+0x72/0x550
2014-01-18T13:50:51.600165+00:00 bl-cassoop-p11 kernel: [74823.656859]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:51.600167+00:00 bl-cassoop-p11 kernel: [74823.656862]  [<ffffffff81051089>] ? default_spin_lock_flags+0x9/0x10
2014-01-18T13:50:51.600168+00:00 bl-cassoop-p11 kernel: [74823.656865]  [<ffffffff817477de>] ? _raw_spin_lock_irqsave+0x2e/0x40
2014-01-18T13:50:51.600170+00:00 bl-cassoop-p11 kernel: [74823.656868]  [<ffffffff81172c36>] handle_pte_fault+0x96/0x230
2014-01-18T13:50:51.600172+00:00 bl-cassoop-p11 kernel: [74823.656871]  [<ffffffff8108c832>] ? hrtimer_cancel+0x22/0x30
2014-01-18T13:50:51.600174+00:00 bl-cassoop-p11 kernel: [74823.656874]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:51.600175+00:00 bl-cassoop-p11 kernel: [74823.656877]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:51.600177+00:00 bl-cassoop-p11 kernel: [74823.656881]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:51.600178+00:00 bl-cassoop-p11 kernel: [74823.656884]  [<ffffffff810c8503>] ? futex_wake+0x113/0x130
2014-01-18T13:50:51.600181+00:00 bl-cassoop-p11 kernel: [74823.656887]  [<ffffffff8108cee4>] ? hrtimer_start_range_ns+0x14/0x20
2014-01-18T13:50:51.600182+00:00 bl-cassoop-p11 kernel: [74823.656891]  [<ffffffff810ca0c8>] ? do_futex+0xd8/0x1b0
2014-01-18T13:50:51.600184+00:00 bl-cassoop-p11 kernel: [74823.656894]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:51.600185+00:00 bl-cassoop-p11 kernel: [74823.656897]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:51.600188+00:00 bl-cassoop-p11 kernel: [74823.656899] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:50:51.624044+00:00 bl-cassoop-p11 kernel: [74823.678646] BUG: soft lockup - CPU#16 stuck for 22s! [kworker/u97:9:42017]
2014-01-18T13:50:51.624052+00:00 bl-cassoop-p11 kernel: [74823.680724] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.624056+00:00 bl-cassoop-p11 kernel: [74823.680751] CPU: 16 PID: 42017 Comm: kworker/u97:9 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.624057+00:00 bl-cassoop-p11 kernel: [74823.680752] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.624059+00:00 bl-cassoop-p11 kernel: [74823.680756] Workqueue: writeback bdi_writeback_workfn (flush-252:24)
2014-01-18T13:50:51.624060+00:00 bl-cassoop-p11 kernel: [74823.680758] task: ffff8817f8ef4650 ti: ffff880075d0a000 task.ti: ffff880075d0a000
2014-01-18T13:50:51.624063+00:00 bl-cassoop-p11 kernel: [74823.680760] RIP: 0010:[<ffffffff810cb99e>]  [<ffffffff810cb99e>] smp_call_function_many+0x26e/0x2d0
2014-01-18T13:50:51.624064+00:00 bl-cassoop-p11 kernel: [74823.680764] RSP: 0018:ffff880075d0b638  EFLAGS: 00000202
2014-01-18T13:50:51.624066+00:00 bl-cassoop-p11 kernel: [74823.680765] RAX: 0000000000000008 RBX: ffffffff81d03da0 RCX: ffff88181fa98720
2014-01-18T13:50:51.624067+00:00 bl-cassoop-p11 kernel: [74823.680767] RDX: 0000000000000008 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.624069+00:00 bl-cassoop-p11 kernel: [74823.680768] RBP: ffff880075d0b698 R08: ffff88181fb15088 R09: 0000000000000000
2014-01-18T13:50:51.624070+00:00 bl-cassoop-p11 kernel: [74823.680770] R10: 0000000000000004 R11: 0000000000000210 R12: ffffffff81254cb7
2014-01-18T13:50:51.624072+00:00 bl-cassoop-p11 kernel: [74823.680772] R13: ffff880075d0b5c8 R14: 0000000000000292 R15: ffff880075d0b5d8
2014-01-18T13:50:51.624073+00:00 bl-cassoop-p11 kernel: [74823.680774] FS:  0000000000000000(0000) GS:ffff88181fb00000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.624075+00:00 bl-cassoop-p11 kernel: [74823.680776] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.624076+00:00 bl-cassoop-p11 kernel: [74823.680777] CR2: 00000006bdbc8000 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:51.624077+00:00 bl-cassoop-p11 kernel: [74823.680779] Stack:
2014-01-18T13:50:51.624079+00:00 bl-cassoop-p11 kernel: [74823.680780]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98720 ffffffff00000008
2014-01-18T13:50:51.624080+00:00 bl-cassoop-p11 kernel: [74823.680785]  01ff88005e6f34e0 ffff88181fb150a8 ffff881500000004 ffff882ff7e6de80
2014-01-18T13:50:51.624082+00:00 bl-cassoop-p11 kernel: [74823.680789]  ffff882ff7e6e130 00007f5918b9e000 ffff882ff7e6de80 00007f5918b9e000
2014-01-18T13:50:51.624083+00:00 bl-cassoop-p11 kernel: [74823.680794] Call Trace:
2014-01-18T13:50:51.624084+00:00 bl-cassoop-p11 kernel: [74823.680798]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.624086+00:00 bl-cassoop-p11 kernel: [74823.680802]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.624088+00:00 bl-cassoop-p11 kernel: [74823.680805]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:51.624089+00:00 bl-cassoop-p11 kernel: [74823.680808]  [<ffffffff8117d33e>] page_mkclean_one+0x9e/0xd0
2014-01-18T13:50:51.624091+00:00 bl-cassoop-p11 kernel: [74823.680811]  [<ffffffff814d9d50>] ? scsi_finish_command+0x120/0x120
2014-01-18T13:50:51.624093+00:00 bl-cassoop-p11 kernel: [74823.680815]  [<ffffffff8117d40a>] page_mkclean_file+0x9a/0xf0
2014-01-18T13:50:51.624094+00:00 bl-cassoop-p11 kernel: [74823.680818]  [<ffffffff8117d49e>] page_mkclean+0x3e/0x50
2014-01-18T13:50:51.624096+00:00 bl-cassoop-p11 kernel: [74823.680821]  [<ffffffff81153268>] clear_page_dirty_for_io+0x58/0xe0
2014-01-18T13:50:51.624098+00:00 bl-cassoop-p11 kernel: [74823.680825]  [<ffffffff8124fd02>] mpage_submit_page+0x52/0x90
2014-01-18T13:50:51.624100+00:00 bl-cassoop-p11 kernel: [74823.680828]  [<ffffffff8124fdee>] mpage_process_page_bufs+0xae/0xc0
2014-01-18T13:50:51.624101+00:00 bl-cassoop-p11 kernel: [74823.680832]  [<ffffffff812510dc>] mpage_prepare_extent_to_map+0x21c/0x300
2014-01-18T13:50:51.624103+00:00 bl-cassoop-p11 kernel: [74823.680835]  [<ffffffff8129ff9d>] ? jbd2__journal_start+0x4d/0x60
2014-01-18T13:50:51.624105+00:00 bl-cassoop-p11 kernel: [74823.680839]  [<ffffffff8127dc5c>] ? __ext4_journal_start_sb+0x7c/0x100
2014-01-18T13:50:51.624106+00:00 bl-cassoop-p11 kernel: [74823.680842]  [<ffffffff81257170>] ? ext4_writepages+0x320/0x750
2014-01-18T13:50:51.624108+00:00 bl-cassoop-p11 kernel: [74823.680846]  [<ffffffff812571a9>] ext4_writepages+0x359/0x750
2014-01-18T13:50:51.624110+00:00 bl-cassoop-p11 kernel: [74823.680849]  [<ffffffff81154930>] do_writepages+0x20/0x40
2014-01-18T13:50:51.624111+00:00 bl-cassoop-p11 kernel: [74823.680853]  [<ffffffff811dc869>] __writeback_single_inode+0x49/0x1d0
2014-01-18T13:50:51.624113+00:00 bl-cassoop-p11 kernel: [74823.680856]  [<ffffffff811ddc40>] writeback_sb_inodes+0x1b0/0x360
2014-01-18T13:50:51.624115+00:00 bl-cassoop-p11 kernel: [74823.680859]  [<ffffffff811dde8e>] __writeback_inodes_wb+0x9e/0xd0
2014-01-18T13:50:51.624116+00:00 bl-cassoop-p11 kernel: [74823.680862]  [<ffffffff811de14b>] wb_writeback+0x28b/0x330
2014-01-18T13:50:51.624118+00:00 bl-cassoop-p11 kernel: [74823.680866]  [<ffffffff811de34a>] wb_do_writeback+0xaa/0x1c0
2014-01-18T13:50:51.624119+00:00 bl-cassoop-p11 kernel: [74823.680869]  [<ffffffff8108445f>] ? set_worker_desc+0x6f/0x80
2014-01-18T13:50:51.624121+00:00 bl-cassoop-p11 kernel: [74823.680872]  [<ffffffff811dfc68>] bdi_writeback_workfn+0x78/0x200
2014-01-18T13:50:51.624123+00:00 bl-cassoop-p11 kernel: [74823.680876]  [<ffffffff81081060>] process_one_work+0x170/0x4a0
2014-01-18T13:50:51.624124+00:00 bl-cassoop-p11 kernel: [74823.680879]  [<ffffffff81082121>] worker_thread+0x121/0x390
2014-01-18T13:50:51.624126+00:00 bl-cassoop-p11 kernel: [74823.680882]  [<ffffffff81082000>] ? manage_workers.isra.21+0x170/0x170
2014-01-18T13:50:51.624127+00:00 bl-cassoop-p11 kernel: [74823.680885]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:51.624129+00:00 bl-cassoop-p11 kernel: [74823.680889]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.624131+00:00 bl-cassoop-p11 kernel: [74823.680893]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:51.624132+00:00 bl-cassoop-p11 kernel: [74823.680896]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.624144+00:00 bl-cassoop-p11 kernel: [74823.680897] Code: 48 63 d0 e8 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 <f6> 41 20 01 74 c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8
2014-01-18T13:50:51.635957+00:00 bl-cassoop-p11 kernel: [74823.690648] BUG: soft lockup - CPU#17 stuck for 22s! [java:42521]
2014-01-18T13:50:51.635965+00:00 bl-cassoop-p11 kernel: [74823.692674] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.635968+00:00 bl-cassoop-p11 kernel: [74823.692696] CPU: 17 PID: 42521 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.635970+00:00 bl-cassoop-p11 kernel: [74823.692697] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.635971+00:00 bl-cassoop-p11 kernel: [74823.692699] task: ffff882fedf65dc0 ti: ffff882feccbe000 task.ti: ffff882feccbe000
2014-01-18T13:50:51.635974+00:00 bl-cassoop-p11 kernel: [74823.692701] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:50:51.635975+00:00 bl-cassoop-p11 kernel: [74823.692704] RSP: 0000:ffff882feccbf998  EFLAGS: 00000202
2014-01-18T13:50:51.635977+00:00 bl-cassoop-p11 kernel: [74823.692706] RAX: 0000000000000006 RBX: ffffffff81d03da0 RCX: ffff88181fa78748
2014-01-18T13:50:51.635978+00:00 bl-cassoop-p11 kernel: [74823.692707] RDX: 0000000000000006 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.635980+00:00 bl-cassoop-p11 kernel: [74823.692709] RBP: ffff882feccbf9f8 R08: ffff88301f315088 R09: 0000000000000000
2014-01-18T13:50:51.635981+00:00 bl-cassoop-p11 kernel: [74823.692711] R10: 0000000000000004 R11: 0000000000017960 R12: ffff880cff722548
2014-01-18T13:50:51.635983+00:00 bl-cassoop-p11 kernel: [74823.692712] R13: 0000000000000000 R14: 0000000000000292 R15: ffff882feccbf938
2014-01-18T13:50:51.635984+00:00 bl-cassoop-p11 kernel: [74823.692714] FS:  00007f58df7f7700(0000) GS:ffff88301f300000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.635986+00:00 bl-cassoop-p11 kernel: [74823.692716] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.635987+00:00 bl-cassoop-p11 kernel: [74823.692718] CR2: 00007f589f160248 CR3: 0000002ff6d16000 CR4: 00000000000407e0
2014-01-18T13:50:51.635988+00:00 bl-cassoop-p11 kernel: [74823.692719] Stack:
2014-01-18T13:50:51.635990+00:00 bl-cassoop-p11 kernel: [74823.692720]  ffff88181fa95050 ffff88181fa95040 ffff88181fa98748 0000000000000008
2014-01-18T13:50:51.635991+00:00 bl-cassoop-p11 kernel: [74823.692726]  0100000000000000 ffff88301f3150a8 ffffffff00000000 ffff882ff7e6de80
2014-01-18T13:50:51.635993+00:00 bl-cassoop-p11 kernel: [74823.692731]  ffff882ff7e6e130 00007f589f160000 ffff882ff7e6de80 ffffea004c94e500
2014-01-18T13:50:51.635994+00:00 bl-cassoop-p11 kernel: [74823.692735] Call Trace:
2014-01-18T13:50:51.635996+00:00 bl-cassoop-p11 kernel: [74823.692739]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.635997+00:00 bl-cassoop-p11 kernel: [74823.692743]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.635999+00:00 bl-cassoop-p11 kernel: [74823.692746]  [<ffffffff811833f3>] ptep_clear_flush+0x43/0x60
2014-01-18T13:50:51.636000+00:00 bl-cassoop-p11 kernel: [74823.692750]  [<ffffffff8117e080>] try_to_unmap_one+0xf0/0x490
2014-01-18T13:50:51.636002+00:00 bl-cassoop-p11 kernel: [74823.692753]  [<ffffffff8117eee8>] try_to_unmap_anon+0xa8/0x130
2014-01-18T13:50:51.636004+00:00 bl-cassoop-p11 kernel: [74823.692757]  [<ffffffff8117f015>] try_to_unmap+0x55/0x70
2014-01-18T13:50:51.636005+00:00 bl-cassoop-p11 kernel: [74823.692761]  [<ffffffff811a0596>] __unmap_and_move+0x116/0x290
2014-01-18T13:50:51.636007+00:00 bl-cassoop-p11 kernel: [74823.692764]  [<ffffffff811a0786>] unmap_and_move+0x76/0x180
2014-01-18T13:50:51.636008+00:00 bl-cassoop-p11 kernel: [74823.692768]  [<ffffffff811a0a91>] migrate_pages+0xa1/0x1e0
2014-01-18T13:50:51.636010+00:00 bl-cassoop-p11 kernel: [74823.692772]  [<ffffffff8119efb0>] ? ftrace_raw_event_mm_migrate_pages+0xe0/0xe0
2014-01-18T13:50:51.636012+00:00 bl-cassoop-p11 kernel: [74823.692775]  [<ffffffff811a1584>] migrate_misplaced_page+0xb4/0xe0
2014-01-18T13:50:51.636013+00:00 bl-cassoop-p11 kernel: [74823.692778]  [<ffffffff81172b43>] do_numa_page+0xd3/0x130
2014-01-18T13:50:51.636015+00:00 bl-cassoop-p11 kernel: [74823.692781]  [<ffffffff81172d4c>] handle_pte_fault+0x1ac/0x230
2014-01-18T13:50:51.636017+00:00 bl-cassoop-p11 kernel: [74823.692785]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:50:51.636018+00:00 bl-cassoop-p11 kernel: [74823.692789]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:51.636020+00:00 bl-cassoop-p11 kernel: [74823.692792]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:51.636021+00:00 bl-cassoop-p11 kernel: [74823.692795]  [<ffffffff811d32f9>] ? mntput_no_expire+0x49/0x160
2014-01-18T13:50:51.636023+00:00 bl-cassoop-p11 kernel: [74823.692799]  [<ffffffff811d3434>] ? mntput+0x24/0x40
2014-01-18T13:50:51.636024+00:00 bl-cassoop-p11 kernel: [74823.692802]  [<ffffffff811b5749>] ? __fput+0x189/0x240
2014-01-18T13:50:51.636026+00:00 bl-cassoop-p11 kernel: [74823.692806]  [<ffffffff811b584e>] ? ____fput+0xe/0x10
2014-01-18T13:50:51.636027+00:00 bl-cassoop-p11 kernel: [74823.692809]  [<ffffffff8108584d>] ? task_work_run+0xcd/0xf0
2014-01-18T13:50:51.636029+00:00 bl-cassoop-p11 kernel: [74823.692813]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:51.636030+00:00 bl-cassoop-p11 kernel: [74823.692817]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:51.636033+00:00 bl-cassoop-p11 kernel: [74823.692819] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:50:51.648001+00:00 bl-cassoop-p11 kernel: [74823.702655] BUG: soft lockup - CPU#18 stuck for 23s! [kswapd0:283]
2014-01-18T13:50:51.648009+00:00 bl-cassoop-p11 kernel: [74823.704696] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.648012+00:00 bl-cassoop-p11 kernel: [74823.704722] CPU: 18 PID: 283 Comm: kswapd0 Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.648014+00:00 bl-cassoop-p11 kernel: [74823.704724] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.648015+00:00 bl-cassoop-p11 kernel: [74823.704726] task: ffff882ff9c41770 ti: ffff882ff8360000 task.ti: ffff882ff8360000
2014-01-18T13:50:51.648018+00:00 bl-cassoop-p11 kernel: [74823.704727] RIP: 0010:[<ffffffff810cb4a2>]  [<ffffffff810cb4a2>] generic_exec_single+0x82/0xb0
2014-01-18T13:50:51.648019+00:00 bl-cassoop-p11 kernel: [74823.704731] RSP: 0018:ffff882ff83616e8  EFLAGS: 00000202
2014-01-18T13:50:51.648021+00:00 bl-cassoop-p11 kernel: [74823.704732] RAX: 0000000000000282 RBX: 0000000000000000 RCX: 0000000000000001
2014-01-18T13:50:51.648022+00:00 bl-cassoop-p11 kernel: [74823.704734] RDX: ffff88181fa386a8 RSI: 0000000000000282 RDI: 0000000000000282
2014-01-18T13:50:51.648024+00:00 bl-cassoop-p11 kernel: [74823.704736] RBP: ffff882ff8361728 R08: ffff882ff7e68650 R09: 0000000000000100
2014-01-18T13:50:51.648025+00:00 bl-cassoop-p11 kernel: [74823.704737] R10: 0000000000000001 R11: 0000000000000400 R12: ffffffff810a2fcc
2014-01-18T13:50:51.648027+00:00 bl-cassoop-p11 kernel: [74823.704739] R13: ffff882ff8361698 R14: ffff8817f2191200 R15: ffff882ff8361648
2014-01-18T13:50:51.648028+00:00 bl-cassoop-p11 kernel: [74823.704741] FS:  0000000000000000(0000) GS:ffff88181fb20000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.648030+00:00 bl-cassoop-p11 kernel: [74823.704743] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.648031+00:00 bl-cassoop-p11 kernel: [74823.704745] CR2: 00007f3b30de2420 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:51.648032+00:00 bl-cassoop-p11 kernel: [74823.704746] Stack:
2014-01-18T13:50:51.648034+00:00 bl-cassoop-p11 kernel: [74823.704747]  ffff882ff8361728 ffff88181fa386a8 02fc000000000000 0000000000000002
2014-01-18T13:50:51.648035+00:00 bl-cassoop-p11 kernel: [74823.704752]  ffffffff8105ae90 0000000000000012 ffffffff81d03da0 0000000000000012
2014-01-18T13:50:51.648037+00:00 bl-cassoop-p11 kernel: [74823.704756]  ffff882ff8361798 ffffffff810cb5a5 0000000000000000 ffff88181fb37b88
2014-01-18T13:50:51.648045+00:00 bl-cassoop-p11 kernel: [74823.704761] Call Trace:
2014-01-18T13:50:51.648047+00:00 bl-cassoop-p11 kernel: [74823.704765]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.648048+00:00 bl-cassoop-p11 kernel: [74823.704768]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:51.648050+00:00 bl-cassoop-p11 kernel: [74823.704772]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.648052+00:00 bl-cassoop-p11 kernel: [74823.704775]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.648053+00:00 bl-cassoop-p11 kernel: [74823.704778]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:51.648055+00:00 bl-cassoop-p11 kernel: [74823.704781]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.648057+00:00 bl-cassoop-p11 kernel: [74823.704785]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.648058+00:00 bl-cassoop-p11 kernel: [74823.704789]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:51.648060+00:00 bl-cassoop-p11 kernel: [74823.704792]  [<ffffffff8117d5ee>] page_referenced_one+0x9e/0x1b0
2014-01-18T13:50:51.648061+00:00 bl-cassoop-p11 kernel: [74823.704796]  [<ffffffff8119a1d1>] ? kmem_cache_free+0x121/0x180
2014-01-18T13:50:51.648063+00:00 bl-cassoop-p11 kernel: [74823.704799]  [<ffffffff811660d9>] ? __mod_zone_page_state+0x49/0x50
2014-01-18T13:50:51.648065+00:00 bl-cassoop-p11 kernel: [74823.704802]  [<ffffffff8117d7e4>] page_referenced_file+0xe4/0x150
2014-01-18T13:50:51.648066+00:00 bl-cassoop-p11 kernel: [74823.704806]  [<ffffffff8117f1e5>] page_referenced+0x65/0xf0
2014-01-18T13:50:51.648068+00:00 bl-cassoop-p11 kernel: [74823.704809]  [<ffffffff81159472>] page_check_references.isra.38+0x22/0xa0
2014-01-18T13:50:51.648070+00:00 bl-cassoop-p11 kernel: [74823.704813]  [<ffffffff8115bd83>] shrink_page_list+0x3f3/0x740
2014-01-18T13:50:51.648071+00:00 bl-cassoop-p11 kernel: [74823.704816]  [<ffffffff810a0fe3>] ? update_curr+0x143/0x200
2014-01-18T13:50:51.648073+00:00 bl-cassoop-p11 kernel: [74823.704819]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:51.648075+00:00 bl-cassoop-p11 kernel: [74823.704823]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:51.648076+00:00 bl-cassoop-p11 kernel: [74823.704826]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:51.648078+00:00 bl-cassoop-p11 kernel: [74823.704830]  [<ffffffff810da6b9>] ? cgroup_next_descendant_pre+0x69/0x80
2014-01-18T13:50:51.648080+00:00 bl-cassoop-p11 kernel: [74823.704833]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:51.648081+00:00 bl-cassoop-p11 kernel: [74823.704837]  [<ffffffff8115d723>] kswapd_shrink_zone+0xe3/0x1a0
2014-01-18T13:50:51.648083+00:00 bl-cassoop-p11 kernel: [74823.704840]  [<ffffffff8115e50c>] balance_pgdat+0x32c/0x470
2014-01-18T13:50:51.648092+00:00 bl-cassoop-p11 kernel: [74823.704843]  [<ffffffff81166610>] ? vm_events_fold_cpu+0x50/0x50
2014-01-18T13:50:51.648093+00:00 bl-cassoop-p11 kernel: [74823.704847]  [<ffffffff8115e773>] kswapd+0x123/0x240
2014-01-18T13:50:51.648095+00:00 bl-cassoop-p11 kernel: [74823.704850]  [<ffffffff8115e650>] ? balance_pgdat+0x470/0x470
2014-01-18T13:50:51.648096+00:00 bl-cassoop-p11 kernel: [74823.704853]  [<ffffffff81088fe0>] kthread+0xc0/0xd0
2014-01-18T13:50:51.648098+00:00 bl-cassoop-p11 kernel: [74823.704857]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.648099+00:00 bl-cassoop-p11 kernel: [74823.704861]  [<ffffffff817508ac>] ret_from_fork+0x7c/0xb0
2014-01-18T13:50:51.648101+00:00 bl-cassoop-p11 kernel: [74823.704864]  [<ffffffff81088f20>] ? flush_kthread_worker+0xb0/0xb0
2014-01-18T13:50:51.648106+00:00 bl-cassoop-p11 kernel: [74823.704865] Code: 8b 55 08 49 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 <f6> 43 20 01 75 f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75
2014-01-18T13:50:51.671868+00:00 bl-cassoop-p11 kernel: [74823.726653] BUG: soft lockup - CPU#20 stuck for 23s! [jsvc:42019]
2014-01-18T13:50:51.671877+00:00 bl-cassoop-p11 kernel: [74823.728699] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.671880+00:00 bl-cassoop-p11 kernel: [74823.728715] CPU: 20 PID: 42019 Comm: jsvc Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.671882+00:00 bl-cassoop-p11 kernel: [74823.728716] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.671883+00:00 bl-cassoop-p11 kernel: [74823.728717] task: ffff880045f58000 ti: ffff8801111b2000 task.ti: ffff8801111b2000
2014-01-18T13:50:51.671885+00:00 bl-cassoop-p11 kernel: [74823.728719] RIP: 0010:[<ffffffff810cb9a2>]  [<ffffffff810cb9a2>] smp_call_function_many+0x272/0x2d0
2014-01-18T13:50:51.671887+00:00 bl-cassoop-p11 kernel: [74823.728722] RSP: 0018:ffff8801111b3cd8  EFLAGS: 00000202
2014-01-18T13:50:51.671888+00:00 bl-cassoop-p11 kernel: [74823.728723] RAX: 0000000000000002 RBX: ffffffff81d03da0 RCX: ffff88181fa387c0
2014-01-18T13:50:51.671890+00:00 bl-cassoop-p11 kernel: [74823.728724] RDX: 0000000000000002 RSI: 0000000000000100 RDI: 0000000000000100
2014-01-18T13:50:51.671892+00:00 bl-cassoop-p11 kernel: [74823.728725] RBP: ffff8801111b3d38 R08: ffff88181fb55090 R09: 0000000000000000
2014-01-18T13:50:51.671893+00:00 bl-cassoop-p11 kernel: [74823.728726] R10: 0000000000000004 R11: 0000000000000206 R12: ffffffff81166a93
2014-01-18T13:50:51.671895+00:00 bl-cassoop-p11 kernel: [74823.728727] R13: ffff8801111b3c48 R14: 0000000000000292 R15: ffff8801111b3c78
2014-01-18T13:50:51.671896+00:00 bl-cassoop-p11 kernel: [74823.728729] FS:  00007efd7a5ea700(0000) GS:ffff88181fb40000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.671898+00:00 bl-cassoop-p11 kernel: [74823.728730] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.671899+00:00 bl-cassoop-p11 kernel: [74823.728731] CR2: 00000006dc026000 CR3: 0000002fee8f1000 CR4: 00000000000407e0
2014-01-18T13:50:51.671900+00:00 bl-cassoop-p11 kernel: [74823.728732] Stack:
2014-01-18T13:50:51.671902+00:00 bl-cassoop-p11 kernel: [74823.728733]  ffff88181fad5050 ffff88181fad5040 ffff88181fad87c0 ffff882f0000000c
2014-01-18T13:50:51.671903+00:00 bl-cassoop-p11 kernel: [74823.728736]  01ff8801111b3d08 ffff88181fb550a8 ffff8801111b3d28 ffff882ff7e68630
2014-01-18T13:50:51.671905+00:00 bl-cassoop-p11 kernel: [74823.728739]  ffff882ff7e68380 00007efd7a5ad000 0000000000000000 ffff882ff7e68380
2014-01-18T13:50:51.671906+00:00 bl-cassoop-p11 kernel: [74823.728742] Call Trace:
2014-01-18T13:50:51.671908+00:00 bl-cassoop-p11 kernel: [74823.728746]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.671910+00:00 bl-cassoop-p11 kernel: [74823.728748]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:51.671911+00:00 bl-cassoop-p11 kernel: [74823.728751]  [<ffffffff8116f362>] tlb_flush_mmu+0x92/0xa0
2014-01-18T13:50:51.671913+00:00 bl-cassoop-p11 kernel: [74823.728754]  [<ffffffff8116f384>] tlb_finish_mmu+0x14/0x40
2014-01-18T13:50:51.671922+00:00 bl-cassoop-p11 kernel: [74823.728756]  [<ffffffff81172152>] zap_page_range+0x122/0x160
2014-01-18T13:50:51.671923+00:00 bl-cassoop-p11 kernel: [74823.728758]  [<ffffffff8116e163>] madvise_vma+0x83/0xc0
2014-01-18T13:50:51.671925+00:00 bl-cassoop-p11 kernel: [74823.728761]  [<ffffffff81178db7>] ? find_vma_prev+0x17/0x60
2014-01-18T13:50:51.671926+00:00 bl-cassoop-p11 kernel: [74823.728763]  [<ffffffff8116e308>] SyS_madvise+0x168/0x280
2014-01-18T13:50:51.671928+00:00 bl-cassoop-p11 kernel: [74823.728766]  [<ffffffff8175095d>] system_call_fastpath+0x1a/0x1f
2014-01-18T13:50:51.671931+00:00 bl-cassoop-p11 kernel: [74823.728767] Code: 95 a2 2b 00 3b 05 03 96 c3 00 0f 8d 1a fe ff ff 48 63 d0 49 8b 0c 24 48 03 0c d5 a0 30 d0 81 f6 41 20 01 74 cc f3 90 f6 41 20 01 <74> c4 eb f6 0f b6 4d c7 4c 89 f2 4c 89 ee 89 df e8 19 fb ff ff
2014-01-18T13:50:51.755953+00:00 bl-cassoop-p11 kernel: [74823.810683] BUG: soft lockup - CPU#27 stuck for 23s! [java:42434]
2014-01-18T13:50:51.755962+00:00 bl-cassoop-p11 kernel: [74823.812713] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.755964+00:00 bl-cassoop-p11 kernel: [74823.812729] CPU: 27 PID: 42434 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.755966+00:00 bl-cassoop-p11 kernel: [74823.812730] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.755968+00:00 bl-cassoop-p11 kernel: [74823.812731] task: ffff8829030e1770 ti: ffff881887d72000 task.ti: ffff881887d72000
2014-01-18T13:50:51.755970+00:00 bl-cassoop-p11 kernel: [74823.812732] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:50:51.755971+00:00 bl-cassoop-p11 kernel: [74823.812735] RSP: 0018:ffff881887d73408  EFLAGS: 00000202
2014-01-18T13:50:51.755973+00:00 bl-cassoop-p11 kernel: [74823.812736] RAX: 0000000000000030 RBX: ffffffff810a3789 RCX: 0000000000000010
2014-01-18T13:50:51.755974+00:00 bl-cassoop-p11 kernel: [74823.812737] RDX: 0000000000000100 RSI: 0000000000000100 RDI: 0000000000000292
2014-01-18T13:50:51.755976+00:00 bl-cassoop-p11 kernel: [74823.812738] RBP: ffff881887d73448 R08: ffffffff8180b540 R09: 0000000000000100
2014-01-18T13:50:51.755977+00:00 bl-cassoop-p11 kernel: [74823.812740] R10: ffffffff811e5a22 R11: 0000000000000000 R12: 0000000000000000
2014-01-18T13:50:51.755979+00:00 bl-cassoop-p11 kernel: [74823.812740] R13: 000000000000000e R14: ffffffff810a108e R15: ffff881887d733b8
2014-01-18T13:50:51.755980+00:00 bl-cassoop-p11 kernel: [74823.812742] FS:  00007f637009a700(0000) GS:ffff88301f3a0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.755982+00:00 bl-cassoop-p11 kernel: [74823.812743] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.755983+00:00 bl-cassoop-p11 kernel: [74823.812744] CR2: 00000000ac432000 CR3: 0000002dd6937000 CR4: 00000000000407e0
2014-01-18T13:50:51.755984+00:00 bl-cassoop-p11 kernel: [74823.812745] Stack:
2014-01-18T13:50:51.755986+00:00 bl-cassoop-p11 kernel: [74823.812746]  0000000000000035 ffff88301f2f5040 ffff881887d73468 000000000000000f
2014-01-18T13:50:51.755987+00:00 bl-cassoop-p11 kernel: [74823.812750]  ffffffff8105ae90 000000000000001b ffffffff81d03da0 000000000000001b
2014-01-18T13:50:51.755989+00:00 bl-cassoop-p11 kernel: [74823.812752]  ffff881887d734b8 ffffffff810cb5a5 ffff881887d73540 0000000000015040
2014-01-18T13:50:51.755990+00:00 bl-cassoop-p11 kernel: [74823.812755] Call Trace:
2014-01-18T13:50:51.755992+00:00 bl-cassoop-p11 kernel: [74823.812758]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.755993+00:00 bl-cassoop-p11 kernel: [74823.812760]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:51.755995+00:00 bl-cassoop-p11 kernel: [74823.812763]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.755996+00:00 bl-cassoop-p11 kernel: [74823.812765]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.755998+00:00 bl-cassoop-p11 kernel: [74823.812767]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:51.756000+00:00 bl-cassoop-p11 kernel: [74823.812770]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.756001+00:00 bl-cassoop-p11 kernel: [74823.812772]  [<ffffffff8105b2f1>] flush_tlb_page+0x61/0xb0
2014-01-18T13:50:51.756003+00:00 bl-cassoop-p11 kernel: [74823.812775]  [<ffffffff81059e02>] ptep_clear_flush_young+0x32/0x40
2014-01-18T13:50:51.756005+00:00 bl-cassoop-p11 kernel: [74823.812777]  [<ffffffff8117e010>] try_to_unmap_one+0x80/0x490
2014-01-18T13:50:51.756006+00:00 bl-cassoop-p11 kernel: [74823.812780]  [<ffffffff8116cee2>] ? vma_interval_tree_iter_first+0x22/0x30
2014-01-18T13:50:51.756008+00:00 bl-cassoop-p11 kernel: [74823.812782]  [<ffffffff8117e4ea>] try_to_unmap_file+0xca/0x2c0
2014-01-18T13:50:51.756010+00:00 bl-cassoop-p11 kernel: [74823.812785]  [<ffffffff8117f1e5>] ? page_referenced+0x65/0xf0
2014-01-18T13:50:51.756011+00:00 bl-cassoop-p11 kernel: [74823.812787]  [<ffffffff8117eff1>] try_to_unmap+0x31/0x70
2014-01-18T13:50:51.756013+00:00 bl-cassoop-p11 kernel: [74823.812790]  [<ffffffff8115bc58>] shrink_page_list+0x2c8/0x740
2014-01-18T13:50:51.756015+00:00 bl-cassoop-p11 kernel: [74823.812792]  [<ffffffff8115c722>] shrink_inactive_list+0x1a2/0x540
2014-01-18T13:50:51.756017+00:00 bl-cassoop-p11 kernel: [74823.812795]  [<ffffffff8115d0d4>] shrink_lruvec+0x274/0x3a0
2014-01-18T13:50:51.756018+00:00 bl-cassoop-p11 kernel: [74823.812797]  [<ffffffff8115d266>] shrink_zone+0x66/0x1a0
2014-01-18T13:50:51.756020+00:00 bl-cassoop-p11 kernel: [74823.812800]  [<ffffffff8115d86d>] shrink_zones+0x8d/0x200
2014-01-18T13:50:51.756021+00:00 bl-cassoop-p11 kernel: [74823.812803]  [<ffffffff8101b569>] ? read_tsc+0x9/0x20
2014-01-18T13:50:51.756023+00:00 bl-cassoop-p11 kernel: [74823.812805]  [<ffffffff810bc73c>] ? ktime_get_ts+0x4c/0xe0
2014-01-18T13:50:51.756024+00:00 bl-cassoop-p11 kernel: [74823.812808]  [<ffffffff8115da6e>] do_try_to_free_pages+0x8e/0x290
2014-01-18T13:50:51.756026+00:00 bl-cassoop-p11 kernel: [74823.812811]  [<ffffffff8115de12>] try_to_free_mem_cgroup_pages+0xd2/0x180
2014-01-18T13:50:51.756028+00:00 bl-cassoop-p11 kernel: [74823.812813]  [<ffffffff817456b6>] ? _cond_resched+0x16/0x40
2014-01-18T13:50:51.756030+00:00 bl-cassoop-p11 kernel: [74823.812817]  [<ffffffff811aa7be>] mem_cgroup_reclaim+0x4e/0xe0
2014-01-18T13:50:51.756031+00:00 bl-cassoop-p11 kernel: [74823.812819]  [<ffffffff811aadba>] mem_cgroup_do_charge+0xea/0x180
2014-01-18T13:50:51.756033+00:00 bl-cassoop-p11 kernel: [74823.812822]  [<ffffffff811aaf61>] __mem_cgroup_try_charge+0x111/0x410
2014-01-18T13:50:51.756035+00:00 bl-cassoop-p11 kernel: [74823.812825]  [<ffffffff811ab887>] mem_cgroup_charge_common+0x57/0xa0
2014-01-18T13:50:51.756036+00:00 bl-cassoop-p11 kernel: [74823.812827]  [<ffffffff811ac19a>] mem_cgroup_newpage_charge+0x2a/0x30
2014-01-18T13:50:51.756038+00:00 bl-cassoop-p11 kernel: [74823.812831]  [<ffffffff8116ea73>] do_anonymous_page.isra.37+0xa3/0x2f0
2014-01-18T13:50:51.756040+00:00 bl-cassoop-p11 kernel: [74823.812833]  [<ffffffff81172da9>] handle_pte_fault+0x209/0x230
2014-01-18T13:50:51.756041+00:00 bl-cassoop-p11 kernel: [74823.812835]  [<ffffffff8174754e>] ? _raw_spin_lock+0xe/0x20
2014-01-18T13:50:51.756043+00:00 bl-cassoop-p11 kernel: [74823.812838]  [<ffffffff8116fc09>] ? __pte_alloc+0xa9/0x160
2014-01-18T13:50:51.756054+00:00 bl-cassoop-p11 kernel: [74823.812840]  [<ffffffff81173ff0>] handle_mm_fault+0x2a0/0x3e0
2014-01-18T13:50:51.756056+00:00 bl-cassoop-p11 kernel: [74823.812842]  [<ffffffff8174b9bf>] __do_page_fault+0x1af/0x560
2014-01-18T13:50:51.756058+00:00 bl-cassoop-p11 kernel: [74823.812846]  [<ffffffff8101260d>] ? __switch_to+0x16d/0x4b0
2014-01-18T13:50:51.756059+00:00 bl-cassoop-p11 kernel: [74823.812849]  [<ffffffff8109fc5d>] ? set_next_entity+0xad/0xd0
2014-01-18T13:50:51.756061+00:00 bl-cassoop-p11 kernel: [74823.812851]  [<ffffffff81095bb2>] ? finish_task_switch+0x52/0xf0
2014-01-18T13:50:51.756062+00:00 bl-cassoop-p11 kernel: [74823.812854]  [<ffffffff817452fa>] ? __schedule+0x3da/0x730
2014-01-18T13:50:51.756064+00:00 bl-cassoop-p11 kernel: [74823.812856]  [<ffffffff8101bc69>] ? sched_clock+0x9/0x10
2014-01-18T13:50:51.756066+00:00 bl-cassoop-p11 kernel: [74823.812859]  [<ffffffff8174bda7>] do_page_fault+0x37/0x70
2014-01-18T13:50:51.756068+00:00 bl-cassoop-p11 kernel: [74823.812861]  [<ffffffff81747e58>] page_fault+0x28/0x30
2014-01-18T13:50:51.756070+00:00 bl-cassoop-p11 kernel: [74823.812863] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T13:50:51.791934+00:00 bl-cassoop-p11 kernel: [74823.846707] BUG: soft lockup - CPU#30 stuck for 23s! [java:36244]
2014-01-18T13:50:51.791943+00:00 bl-cassoop-p11 kernel: [74823.848691] Modules linked in: mpt2sas scsi_transport_sas raid_class mptctl mptbase ipmi_si ipmi_devintf ipmi_msghandler dell_rbu ext2 bonding vesafb dcdbas gpio_ich mei_me shpchp mei joydev sb_edac lpc_ich edac_core wmi mac_hid acpi_power_meter coretemp hid_generic usbhid hid ixgbe tg3 dca ptp mdio pps_core [last unloaded: ipmi_si]
2014-01-18T13:50:51.791946+00:00 bl-cassoop-p11 kernel: [74823.848721] CPU: 30 PID: 36244 Comm: java Tainted: G        W    3.11.0-15-generic #23~precise1-Ubuntu
2014-01-18T13:50:51.791948+00:00 bl-cassoop-p11 kernel: [74823.848723] Hardware name: Dell Inc. PowerEdge R720xd/0X6H47, BIOS 2.1.3 11/20/2013
2014-01-18T13:50:51.791957+00:00 bl-cassoop-p11 kernel: [74823.848725] task: ffff881894a69770 ti: ffff88188471e000 task.ti: ffff88188471e000
2014-01-18T13:50:51.791959+00:00 bl-cassoop-p11 kernel: [74823.848726] RIP: 0010:[<ffffffff810cb4a6>]  [<ffffffff810cb4a6>] generic_exec_single+0x86/0xb0
2014-01-18T13:50:51.791961+00:00 bl-cassoop-p11 kernel: [74823.848729] RSP: 0018:ffff88188471f868  EFLAGS: 00000202
2014-01-18T13:50:51.791962+00:00 bl-cassoop-p11 kernel: [74823.848731] RAX: 0000000000000286 RBX: ffff88187fffcb08 RCX: 0000000000000001
2014-01-18T13:50:51.791964+00:00 bl-cassoop-p11 kernel: [74823.848732] RDX: ffff881887d73470 RSI: 0000000000000286 RDI: 0000000000000286
2014-01-18T13:50:51.791965+00:00 bl-cassoop-p11 kernel: [74823.848734] RBP: ffff88188471f8a8 R08: ffff882ff7b925d0 R09: 0000000000000100
2014-01-18T13:50:51.791967+00:00 bl-cassoop-p11 kernel: [74823.848736] R10: 00000000000035de R11: 0000000000000001 R12: 0000000000000000
2014-01-18T13:50:51.791968+00:00 bl-cassoop-p11 kernel: [74823.848737] R13: ffff88187fffcb00 R14: ffff88187fffcb08 R15: ffffffff00000000
2014-01-18T13:50:51.791970+00:00 bl-cassoop-p11 kernel: [74823.848739] FS:  00007ff4881c1700(0000) GS:ffff88181fbe0000(0000) knlGS:0000000000000000
2014-01-18T13:50:51.791971+00:00 bl-cassoop-p11 kernel: [74823.848741] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2014-01-18T13:50:51.791973+00:00 bl-cassoop-p11 kernel: [74823.848743] CR2: 00007f9151de8880 CR3: 0000000001c0d000 CR4: 00000000000407e0
2014-01-18T13:50:51.791974+00:00 bl-cassoop-p11 kernel: [74823.848744] Stack:
2014-01-18T13:50:51.791975+00:00 bl-cassoop-p11 kernel: [74823.848745]  ffff88284c8d0000 ffff881887d73470 0000000000000003 000000000000000f
2014-01-18T13:50:51.791977+00:00 bl-cassoop-p11 kernel: [74823.848751]  ffffffff8105ae90 000000000000001e ffffffff81d03da0 000000000000001e
2014-01-18T13:50:51.791978+00:00 bl-cassoop-p11 kernel: [74823.848755]  ffff88188471f918 ffffffff810cb5a5 ffff88188471f8d8 ffff88188471f8d8
2014-01-18T13:50:51.791979+00:00 bl-cassoop-p11 kernel: [74823.848760] Call Trace:
2014-01-18T13:50:51.791981+00:00 bl-cassoop-p11 kernel: [74823.848763]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.791983+00:00 bl-cassoop-p11 kernel: [74823.848766]  [<ffffffff810cb5a5>] smp_call_function_single+0xd5/0x160
2014-01-18T13:50:51.791984+00:00 bl-cassoop-p11 kernel: [74823.848770]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.791986+00:00 bl-cassoop-p11 kernel: [74823.848773]  [<ffffffff8105ae90>] ? leave_mm+0x70/0x70
2014-01-18T13:50:51.791987+00:00 bl-cassoop-p11 kernel: [74823.848776]  [<ffffffff810cb9b7>] smp_call_function_many+0x287/0x2d0
2014-01-18T13:50:51.791989+00:00 bl-cassoop-p11 kernel: [74823.848779]  [<ffffffff81733789>] ? memcg_check_events.part.44+0x8c/0xa5
2014-01-18T13:50:51.791991+00:00 bl-cassoop-p11 kernel: [74823.848783]  [<ffffffff8105afee>] native_flush_tlb_others+0x2e/0x30
2014-01-18T13:50:51.791992+00:00 bl-cassoop-p11 kernel: [74823.848786]  [<ffffffff8105b0d0>] flush_tlb_mm_range+0x70/0x230
2014-01-18T13:50:51.792001+00:00 bl-cassoop-p11 kernel: [74823.848790]  [<ffffffff8116f30c>] tlb_flush_mmu+0x3c/0xa0
2014-01-18T13:50:51.792003+00:00 bl-cassoop-p11 kernel: [74823.848793]  [<ffffffff81171190>] zap_pte_range+0x250/0x450
2014-01-18T13:50:51.792004+00:00 bl-cassoop-p11 kernel: [74823.848796]  [<ffffffff81171556>] unmap_page_range+0x1c6/0x320
2014-01-18T13:50:51.792006+00:00 bl-cassoop-p11 kernel: [74823.848799]  [<ffffffff81155a73>] ? __pagevec_lru_add_fn+0x103/0x230
2014-01-18T13:50:51.792008+00:00 bl-cassoop-p11 kernel: [74823.848802]  [<ffffffff81171737>] unmap_single_vma+0x87/0x100
2014-01-18T13:50:51.792009+00:00 bl-cassoop-p11 kernel: [74823.848805]  [<ffffffff81171fe4>] unmap_vmas+0x54/0xa0
2014-01-18T13:50:51.792011+00:00 bl-cassoop-p11 kernel: [74823.848808]  [<ffffffff8117a5cc>] exit_mmap+0x9c/0x170
2014-01-18T13:50:51.792012+00:00 bl-cassoop-p11 kernel: [74823.848811]  [<ffffffff8174621a>] ? io_schedule+0xaa/0xd0
2014-01-18T13:50:51.792014+00:00 bl-cassoop-p11 kernel: [74823.848815]  [<ffffffff81062fea>] mmput.part.22+0x4a/0x110
2014-01-18T13:50:51.792015+00:00 bl-cassoop-p11 kernel: [74823.848818]  [<ffffffff810630d9>] mmput+0x29/0x30
2014-01-18T13:50:51.792017+00:00 bl-cassoop-p11 kernel: [74823.848821]  [<ffffffff81067d46>] exit_mm+0x146/0x190
2014-01-18T13:50:51.792019+00:00 bl-cassoop-p11 kernel: [74823.848824]  [<ffffffff8110b2cb>] ? taskstats_exit+0x1cb/0x270
2014-01-18T13:50:51.792020+00:00 bl-cassoop-p11 kernel: [74823.848828]  [<ffffffff81067ef3>] do_exit+0x163/0x480
2014-01-18T13:50:51.792022+00:00 bl-cassoop-p11 kernel: [74823.848831]  [<ffffffff8107511b>] ? __dequeue_signal+0x6b/0xb0
2014-01-18T13:50:51.792023+00:00 bl-cassoop-p11 kernel: [74823.848834]  [<ffffffff810682a4>] do_group_exit+0x44/0xa0
2014-01-18T13:50:51.792025+00:00 bl-cassoop-p11 kernel: [74823.848837]  [<ffffffff81078281>] get_signal_to_deliver+0x231/0x480
2014-01-18T13:50:51.792027+00:00 bl-cassoop-p11 kernel: [74823.848841]  [<ffffffff81013c57>] do_signal+0x47/0x140
2014-01-18T13:50:51.792028+00:00 bl-cassoop-p11 kernel: [74823.848845]  [<ffffffff811b42e4>] ? vfs_read+0xb4/0x180
2014-01-18T13:50:51.792030+00:00 bl-cassoop-p11 kernel: [74823.848848]  [<ffffffff81013dd8>] do_notify_resume+0x88/0xc0
2014-01-18T13:50:51.792031+00:00 bl-cassoop-p11 kernel: [74823.848851]  [<ffffffff81750c1a>] int_signal+0x12/0x17
2014-01-18T13:50:51.792034+00:00 bl-cassoop-p11 kernel: [74823.848853] Code: 89 5d 08 4c 89 2b 48 89 53 08 48 89 1a e8 d3 c0 67 00 4c 3b 6d c8 74 2b 45 85 ff 75 0a eb 0e 66 0f 1f 44 00 00 f3 90 f6 43 20 01 <75> f8 48 8b 5d d8 4c 8b 65 e0 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d
2014-01-18T14:05:17.377167+00:00 bl-cassoop-p11 kernel: imklog 5.8.6, log source = /proc/kmsg started.
2014-01-18T14:05:17.377556+00:00 bl-cassoop-p11 kernel: [    0.000000] Initializing cgroup subsys cpuset
2014-01-18T14:05:17.377571+00:00 bl-cassoop-p11 kernel: [    0.000000] Initializing cgroup subsys cpu
2014-01-18T14:05:17.377575+00:00 bl-cassoop-p11 kernel: [    0.000000] Initializing cgroup subsys cpuacct
2014-01-18T14:05:17.377628+00:00 bl-cassoop-p11 kernel: [    0.000000] Linux version 3.11.0-15-generic (buildd@tipua) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #23~precise1-Ubuntu SMP Tue Dec 10 16:39:48 UTC 2013 (Ubuntu 3.11.0-15.23~precise1-generic 3.11.10)
2014-01-18T14:05:17.377636+00:00 bl-cassoop-p11 kernel: [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/rootvg-root ro
[12:48]abridgett@apbmac:~$