rendered paste body-----------crm_mon----------------------------
Refresh in 2s...
============
Last updated: Fri Apr 4 13:23:51 2008
Current DC: apolo-slave (30fcbb73-aabd-4d15-8f71-bfb9e538885c)
2 Nodes configured.
3 Resources configured.
============
Node: apolo-slave (30fcbb73-aabd-4d15-8f71-bfb9e538885c): online
Node: apolo-master (23e734e2-b052-429a-b5ae-c3d31c4487a1): online
Master/Slave Set: ms-drbd0
drbd0:0 (heartbeat::ocf:drbd): Started apolo-slave
drbd0:1 (heartbeat::ocf:drbd): Master apolo-master FAILED
Resource Group: group_1
IPaddr_192_168_2_150 (heartbeat::ocf:IPaddr): Started apolo-master
ssh_2 (lsb:ssh): Started apolo-master
Clone Set: stonithclone
stonithclone:0 (stonith:external/ssh): Started apolo-slave
stonithclone:1 (stonith:external/ssh): Started apolo-master
Failed actions:
drbd0:1_promote_0 (node=apolo-master, call=97, rc=1): complete
----------------------------hb log--------------------------
crmd[26524]: 2008/04/04_14:25:27 info: process_lrm_event: LRM operation drbd0:1_demote_0 (call=149, rc=0) complete
crmd[26524]: 2008/04/04_14:25:28 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=73:34:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:28 info: rsc:drbd0:1: notify
crm_master[962]: 2008/04/04_14:25:28 info: Invoked: /usr/sbin/crm_master -v 75 -l reboot
lrmd[26521]: 2008/04/04_14:25:29 info: RA output: (drbd0:1:notify:stdout) No set matching id=master-23e734e2-b052-429a-b5ae-c3d31c4487a1 in status
crmd[26524]: 2008/04/04_14:25:29 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=150, rc=0) complete
crmd[26524]: 2008/04/04_14:25:30 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=67:35:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:30 info: rsc:drbd0:1: notify
crmd[26524]: 2008/04/04_14:25:30 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=151, rc=0) complete
crmd[26524]: 2008/04/04_14:25:32 info: do_lrm_rsc_op: Performing op=drbd0:1_stop_0 key=3:35:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:32 info: rsc:drbd0:1: stop
crm_master[1045]: 2008/04/04_14:25:32 info: Invoked: /usr/sbin/crm_master -D
lrmd[26521]: 2008/04/04_14:25:33 info: RA output: (drbd0:1:stop:stdout)
lrmd[26521]: 2008/04/04_14:25:33 info: RA output: (drbd0:1:stop:stdout)
crmd[26524]: 2008/04/04_14:25:33 info: process_lrm_event: LRM operation drbd0:1_stop_0 (call=152, rc=0) complete
crmd[26524]: 2008/04/04_14:25:34 info: do_lrm_rsc_op: Performing op=drbd0:0_monitor_0 key=8:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:34 info: rsc:drbd0:0: monitor
crmd[26524]: 2008/04/04_14:25:34 info: do_lrm_rsc_op: Performing op=IPaddr_192_168_2_150_start_0 key=40:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:34 info: rsc:IPaddr_192_168_2_150: start
IPaddr[1109]: 2008/04/04_14:25:34 INFO: Using calculated nic for 192.168.2.150: eth0
IPaddr[1109]: 2008/04/04_14:25:34 INFO: Using calculated netmask for 192.168.2.150: 255.255.255.0
crmd[26524]: 2008/04/04_14:25:34 info: process_lrm_event: LRM operation drbd0:0_monitor_0 (call=153, rc=7) complete
IPaddr[1109]: 2008/04/04_14:25:34 INFO: eval ifconfig eth0:0 192.168.2.150 netmask 255.255.255.0 broadcast 192.168.2.255
crmd[26524]: 2008/04/04_14:25:34 info: process_lrm_event: LRM operation IPaddr_192_168_2_150_start_0 (call=154, rc=0) complete
crmd[26524]: 2008/04/04_14:25:36 info: do_lrm_rsc_op: Performing op=drbd0:1_start_0 key=10:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:36 info: rsc:drbd0:1: start
crmd[26524]: 2008/04/04_14:25:36 info: do_lrm_rsc_op: Performing op=IPaddr_192_168_2_150_monitor_5000 key=41:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
crmd[26524]: 2008/04/04_14:25:36 info: do_lrm_rsc_op: Performing op=ssh_2_start_0 key=43:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:36 info: rsc:ssh_2: start
lrmd[1251]: 2008/04/04_14:25:36 WARN: For LSB init script, no additional parameters are needed.
lrmd[26521]: 2008/04/04_14:25:36 info: RA output: (ssh_2:start:stdout) Starting OpenBSD Secure Shell server...
lrmd[26521]: 2008/04/04_14:25:36 info: RA output: (ssh_2:start:stdout) .
crmd[26524]: 2008/04/04_14:25:36 info: process_lrm_event: LRM operation ssh_2_start_0 (call=157, rc=0) complete
crmd[26524]: 2008/04/04_14:25:36 info: process_lrm_event: LRM operation IPaddr_192_168_2_150_monitor_5000 (call=156, rc=0) complete
crmd[26524]: 2008/04/04_14:25:37 info: do_lrm_rsc_op: Performing op=ssh_2_monitor_120000 key=44:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
crmd[26524]: 2008/04/04_14:25:37 info: process_lrm_event: LRM operation ssh_2_monitor_120000 (call=158, rc=0) complete
lrmd[26521]: 2008/04/04_14:25:37 info: RA output: (drbd0:1:start:stdout)
crmd[26524]: 2008/04/04_14:25:37 info: process_lrm_event: LRM operation drbd0:1_start_0 (call=155, rc=0) complete
crmd[26524]: 2008/04/04_14:25:39 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=61:36:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:39 info: rsc:drbd0:1: notify
crm_master[1459]: 2008/04/04_14:25:39 info: Invoked: /usr/sbin/crm_master -v 10 -l reboot
lrmd[26521]: 2008/04/04_14:25:40 info: RA output: (drbd0:1:notify:stdout) No set matching id=master-23e734e2-b052-429a-b5ae-c3d31c4487a1 in status
crmd[26524]: 2008/04/04_14:25:40 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=159, rc=0) complete
crmd[26524]: 2008/04/04_14:25:41 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=62:37:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:41 info: rsc:drbd0:1: notify
crmd[26524]: 2008/04/04_14:25:41 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=160, rc=0) complete
crmd[26524]: 2008/04/04_14:25:42 info: do_lrm_rsc_op: Performing op=drbd0:1_promote_0 key=12:37:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:42 info: rsc:drbd0:1: promote
lrmd[26521]: 2008/04/04_14:25:42 info: RA output: (drbd0:1:promote:stdout) ioctl(,SET_STATE,) failed: Input/output error Local replica is inconsistent (--do-what-I-say ?) Command 'drbdsetup /dev/drbd0 primary' terminated with exit code 21 /bin/bash: -c: option requires an argument Command '/bin/bash -c' terminated with exit code 2 drbdadm aborting
drbd[1492]: 2008/04/04_14:25:42 ERROR: drbd0 promote: Not primary despite drbdadm call.
crmd[26524]: 2008/04/04_14:25:42 info: process_lrm_event: LRM operation drbd0:1_promote_0 (call=161, rc=1) complete
crmd[26524]: 2008/04/04_14:25:43 info: do_lrm_rsc_op: Performing op=ssh_2_stop_0 key=43:38:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:43 info: rsc:ssh_2: stop
lrmd[1601]: 2008/04/04_14:25:43 WARN: For LSB init script, no additional parameters are needed.
crmd[26524]: 2008/04/04_14:25:43 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=72:38:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:43 info: rsc:drbd0:1: notify
crmd[26524]: 2008/04/04_14:25:43 info: process_lrm_event: LRM operation ssh_2_monitor_120000 (call=158, rc=-2) Cancelled
lrmd[26521]: 2008/04/04_14:25:43 info: RA output: (ssh_2:stop:stdout) Stopping OpenBSD Secure Shell server...
lrmd[26521]: 2008/04/04_14:25:43 info: RA output: (ssh_2:stop:stdout) .
crmd[26524]: 2008/04/04_14:25:43 info: process_lrm_event: LRM operation ssh_2_stop_0 (call=162, rc=0) complete
crmd[26524]: 2008/04/04_14:25:43 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=163, rc=0) complete
crmd[26524]: 2008/04/04_14:25:45 info: do_lrm_rsc_op: Performing op=IPaddr_192_168_2_150_stop_0 key=40:38:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:45 info: rsc:IPaddr_192_168_2_150: stop
crmd[26524]: 2008/04/04_14:25:45 info: do_lrm_rsc_op: Performing op=drbd0:1_demote_0 key=14:38:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:45 info: rsc:drbd0:1: demote
crmd[26524]: 2008/04/04_14:25:45 info: process_lrm_event: LRM operation IPaddr_192_168_2_150_monitor_5000 (call=156, rc=-2) Cancelled
lrmd[26521]: 2008/04/04_14:25:45 info: RA output: (IPaddr_192_168_2_150:stop:stdout) In IP Stop
lrmd[26521]: 2008/04/04_14:25:45 info: RA output: (IPaddr_192_168_2_150:stop:stderr) SIOCDELRT: No such process
IPaddr[1634]: 2008/04/04_14:25:45 INFO: ifconfig eth0:0 down
crmd[26524]: 2008/04/04_14:25:45 info: process_lrm_event: LRM operation IPaddr_192_168_2_150_stop_0 (call=164, rc=0) complete
lrmd[26521]: 2008/04/04_14:25:45 info: RA output: (drbd0:1:demote:stdout)
crmd[26524]: 2008/04/04_14:25:45 info: process_lrm_event: LRM operation drbd0:1_demote_0 (call=165, rc=0) complete
crmd[26524]: 2008/04/04_14:25:46 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=73:38:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:46 info: rsc:drbd0:1: notify
crm_master[1736]: 2008/04/04_14:25:46 info: Invoked: /usr/sbin/crm_master -v 75 -l reboot
crmd[26524]: 2008/04/04_14:25:48 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=166, rc=0) complete
lrmd[26521]: 2008/04/04_14:25:48 info: RA output: (drbd0:1:notify:stdout) No set matching id=master-23e734e2-b052-429a-b5ae-c3d31c4487a1 in status
crmd[26524]: 2008/04/04_14:25:49 info: do_lrm_rsc_op: Performing op=drbd0:1_notify_0 key=67:39:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:49 info: rsc:drbd0:1: notify
crmd[26524]: 2008/04/04_14:25:49 info: process_lrm_event: LRM operation drbd0:1_notify_0 (call=167, rc=0) complete
crmd[26524]: 2008/04/04_14:25:50 info: do_lrm_rsc_op: Performing op=drbd0:1_stop_0 key=3:39:2987022a-c9a2-4803-a95c-d3a20c973d23)
lrmd[26521]: 2008/04/04_14:25:50 info: rsc:drbd0:1: stop
crm_master[1818]: 2008/04/04_14:25:50 info: Invoked: /usr/sbin/crm_master -D
lrmd[26521]: 2008/04/04_14:25:51 info: RA output: (drbd0:1:stop:stdout)
lrmd[26521]: 2008/04/04_14:25:51 info: RA output: (drbd0:1:stop:stdout)
crmd[26524]: 2008/04/04_14:25:51 info: process_lrm_event: LRM operation drbd0:1_stop_0 (call=168, rc=0) complete