I have a main site in the US with two S8720s and an ESS site in the
UK with two S8720s and two G650s with IPSIs connecting back to the
US via an MPLS WAN.
I am having a problem getting the UK IPSIs to register and stay registered with the US servers. They are intermittently connecting then getting disconnected and going to ESS mode. When they lose connectivity I cannot ipsisession to them (even from the UK server) but I can ping them from anywhere.
I'm getting a ton of sanity failure messages:
[login to unmask email]> logc lm -t today |grep checkSlot | tail -12
20071205:024931982:8516616:pcd(22448):MED:[[0:0] checkSlot: sanity failure (1)]
20071205:024931982:8516617:pcd(22448):MED:[[0:1] checkSlot: sanity failure (2)]
20071205:024931982:8516618:pcd(22448):MED:[[0:1] checkSlot: too many standby sanity failures (2), active failures (1)]
20071205:024932002:8516623:pcd(22448):MED:[[1:0] checkSlot: sanity failure (1)]
20071205:024932002:8516624:pcd(22448):MED:[[1:1] checkSlot: sanity failure (2)]
20071205:024932002:8516625:pcd(22448):MED:[[1:1] checkSlot: too many standby sanity failures (2), active failures (1)]
20071205:024932982:8516646:pcd(22448):MED:[[0:0] checkSlot: sanity failure (2)]
20071205:024933002:8516647:pcd(22448):MED:[[1:0] checkSlot: sanity failure (2)]
20071205:024933982:8516650:pcd(22448):MED:[[0:0] checkSlot: too many sanity failures (3)]
20071205:024934002:8516655:pcd(22448):MED:[[1:0] checkSlot: too many sanity failures (3)]
20071205:024934982:8516676:pcd(22448):MED:[[0:1] checkSlot: sanity failure (1)]
20071205:024935002:8516677:pcd(22448):MED:[[1:1] checkSlot: sanity failure (1)]
[login to unmask email]>
I also cannot ipsisession to the IPSI boards from anywhere:
From UK S8720
[login to unmask email]> ipsisession -c 3a
System Error: socket operation(connect) timeout.
[login to unmask email]> ipsisession -c 3b
System Error: socket operation(connect) timeout.
From US S8720
[login to unmask email]> ipsisession -c 3a
System Error: socket operation(connect) timeout.
[login to unmask email]> ipsisession -c 3b
System Error: socket operation(connect) timeout.
But....and this is where it gets weird....I have what looks like layer 3 connectivity everywhere. IPSI 3b is 10.20.177.2
From UK S8720
[login to unmask email]> ping 10.20.177.2
PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
64 bytes from 10.20.177.2: icmp_seq=0 ttl=64 time=0.312 ms
64 bytes from 10.20.177.2: icmp_seq=1 ttl=64 time=0.308 ms
64 bytes from 10.20.177.2: icmp_seq=2 ttl=64 time=0.311 ms
From US S8720
[login to unmask email]> ping 10.20.177.2
PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
64 bytes from 10.20.177.2: icmp_seq=0 ttl=58 time=102 ms
64 bytes from 10.20.177.2: icmp_seq=1 ttl=58 time=113 ms
64 bytes from 10.20.177.2: icmp_seq=2 ttl=58 time=120 ms
The data switches at both sites are Cisco 3560s. If I reset one of the UK IPSI boards I get connectivity back to the US momentarily. At that time I can ipsisession successfully and that session will stay up as long as I stay connected. Once I exit out I cannot get back in.
This is all very new to me. Any help would be appreciated.
I am having a problem getting the UK IPSIs to register and stay registered with the US servers. They are intermittently connecting then getting disconnected and going to ESS mode. When they lose connectivity I cannot ipsisession to them (even from the UK server) but I can ping them from anywhere.
I'm getting a ton of sanity failure messages:
[login to unmask email]> logc lm -t today |grep checkSlot | tail -12
20071205:024931982:8516616:pcd(22448):MED:[[0:0] checkSlot: sanity failure (1)]
20071205:024931982:8516617:pcd(22448):MED:[[0:1] checkSlot: sanity failure (2)]
20071205:024931982:8516618:pcd(22448):MED:[[0:1] checkSlot: too many standby sanity failures (2), active failures (1)]
20071205:024932002:8516623:pcd(22448):MED:[[1:0] checkSlot: sanity failure (1)]
20071205:024932002:8516624:pcd(22448):MED:[[1:1] checkSlot: sanity failure (2)]
20071205:024932002:8516625:pcd(22448):MED:[[1:1] checkSlot: too many standby sanity failures (2), active failures (1)]
20071205:024932982:8516646:pcd(22448):MED:[[0:0] checkSlot: sanity failure (2)]
20071205:024933002:8516647:pcd(22448):MED:[[1:0] checkSlot: sanity failure (2)]
20071205:024933982:8516650:pcd(22448):MED:[[0:0] checkSlot: too many sanity failures (3)]
20071205:024934002:8516655:pcd(22448):MED:[[1:0] checkSlot: too many sanity failures (3)]
20071205:024934982:8516676:pcd(22448):MED:[[0:1] checkSlot: sanity failure (1)]
20071205:024935002:8516677:pcd(22448):MED:[[1:1] checkSlot: sanity failure (1)]
[login to unmask email]>
I also cannot ipsisession to the IPSI boards from anywhere:
From UK S8720
[login to unmask email]> ipsisession -c 3a
System Error: socket operation(connect) timeout.
[login to unmask email]> ipsisession -c 3b
System Error: socket operation(connect) timeout.
From US S8720
[login to unmask email]> ipsisession -c 3a
System Error: socket operation(connect) timeout.
[login to unmask email]> ipsisession -c 3b
System Error: socket operation(connect) timeout.
But....and this is where it gets weird....I have what looks like layer 3 connectivity everywhere. IPSI 3b is 10.20.177.2
From UK S8720
[login to unmask email]> ping 10.20.177.2
PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
64 bytes from 10.20.177.2: icmp_seq=0 ttl=64 time=0.312 ms
64 bytes from 10.20.177.2: icmp_seq=1 ttl=64 time=0.308 ms
64 bytes from 10.20.177.2: icmp_seq=2 ttl=64 time=0.311 ms
From US S8720
[login to unmask email]> ping 10.20.177.2
PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
64 bytes from 10.20.177.2: icmp_seq=0 ttl=58 time=102 ms
64 bytes from 10.20.177.2: icmp_seq=1 ttl=58 time=113 ms
64 bytes from 10.20.177.2: icmp_seq=2 ttl=58 time=120 ms
The data switches at both sites are Cisco 3560s. If I reset one of the UK IPSI boards I get connectivity back to the US momentarily. At that time I can ipsisession successfully and that session will stay up as long as I stay connected. Once I exit out I cannot get back in.
This is all very new to me. Any help would be appreciated.