Dec 05
    2007

    IPSI Connectivity - Sanity Failures

    Inactive Forum User
    I have a main site in the US with two S8720s and an ESS site in the UK with two S8720s and two G650s with IPSIs connecting back to the US via an MPLS WAN.

    I am having a problem getting the UK IPSIs to register and stay registered with the US servers. They are intermittently connecting then getting disconnected and going to ESS mode. When they lose connectivity I cannot ipsisession to them (even from the UK server) but I can ping them from anywhere.

    I'm getting a ton of sanity failure messages:

    [login to unmask email]> logc lm -t today |grep checkSlot | tail -12
    20071205:024931982:8516616:pcd(22448):MED:[[0:0] checkSlot: sanity failure (1)]
    20071205:024931982:8516617:pcd(22448):MED:[[0:1] checkSlot: sanity failure (2)]
    20071205:024931982:8516618:pcd(22448):MED:[[0:1] checkSlot: too many standby sanity failures (2), active failures (1)]
    20071205:024932002:8516623:pcd(22448):MED:[[1:0] checkSlot: sanity failure (1)]
    20071205:024932002:8516624:pcd(22448):MED:[[1:1] checkSlot: sanity failure (2)]
    20071205:024932002:8516625:pcd(22448):MED:[[1:1] checkSlot: too many standby sanity failures (2), active failures (1)]
    20071205:024932982:8516646:pcd(22448):MED:[[0:0] checkSlot: sanity failure (2)]
    20071205:024933002:8516647:pcd(22448):MED:[[1:0] checkSlot: sanity failure (2)]
    20071205:024933982:8516650:pcd(22448):MED:[[0:0] checkSlot: too many sanity failures (3)]
    20071205:024934002:8516655:pcd(22448):MED:[[1:0] checkSlot: too many sanity failures (3)]
    20071205:024934982:8516676:pcd(22448):MED:[[0:1] checkSlot: sanity failure (1)]
    20071205:024935002:8516677:pcd(22448):MED:[[1:1] checkSlot: sanity failure (1)]
    [login to unmask email]>


    I also cannot ipsisession to the IPSI boards from anywhere:
    From UK S8720
    [login to unmask email]> ipsisession -c 3a
    System Error: socket operation(connect) timeout.

    [login to unmask email]> ipsisession -c 3b
    System Error: socket operation(connect) timeout.


    From US S8720
    [login to unmask email]> ipsisession -c 3a
    System Error: socket operation(connect) timeout.

    [login to unmask email]> ipsisession -c 3b
    System Error: socket operation(connect) timeout.


    But....and this is where it gets weird....I have what looks like layer 3 connectivity everywhere. IPSI 3b is 10.20.177.2

    From UK S8720
    [login to unmask email]> ping 10.20.177.2
    PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
    64 bytes from 10.20.177.2: icmp_seq=0 ttl=64 time=0.312 ms
    64 bytes from 10.20.177.2: icmp_seq=1 ttl=64 time=0.308 ms
    64 bytes from 10.20.177.2: icmp_seq=2 ttl=64 time=0.311 ms


    From US S8720
    [login to unmask email]> ping 10.20.177.2
    PING 10.20.177.2 (10.20.177.2) 56(84) bytes of data.
    64 bytes from 10.20.177.2: icmp_seq=0 ttl=58 time=102 ms
    64 bytes from 10.20.177.2: icmp_seq=1 ttl=58 time=113 ms
    64 bytes from 10.20.177.2: icmp_seq=2 ttl=58 time=120 ms


    The data switches at both sites are Cisco 3560s. If I reset one of the UK IPSI boards I get connectivity back to the US momentarily. At that time I can ipsisession successfully and that session will stay up as long as I stay connected. Once I exit out I cannot get back in.

    This is all very new to me. Any help would be appreciated.
    Max Barker
    [Avaya]
    I know this is basic but, are the IPSIs and Cisco ports both locked to 100Mb Full Duplex. (Make sure you do not have a duplex mismatch)

    A 'netstat -an | grep 5010' from the linux shell will show you if the sockets to the IPSIs are established or not.
    Max Barker
    [Avaya]
    I just thought of something else...

    Are you running QOS across the WAN? The Cisco's should 'Trust' our IPSI QOS settings and the MPLS prioritize accordingly.

    On CM in the 'ipserver-interface' form make sure you have 'Enable QOS' stroked to 'y' and set your Layer 2/3 values accordingly. Eg. 5 & 46. This will enable QOS from the server to the IPSI, however, it will NOT set it from the IPSI to the server, this must be done on the IPSI itself.

    Login to the IPSIs and do a 'show qos' to see current values. If it does not match your customer's QOS policy, and if default it probably won't, then change to match..

    Use 'set diffserv 46' as this is the norm and reset the IPSI. Log back in and make sure the values are correct.

    Also do a 'show port 1' to make sure you are 'locked down'

    Hope this helps..
    David Hansen
    Also if this connection is running through a firewall, make sure that the firewall isn't stripping the COS / QOS values when it pushes it to the MPLS network.
    Inactive Forum User
    Looks like the sockets are established.

    [login to unmask email]> netstat -an | grep 5010
    tcp 0 40 10.20.206.1:34183 10.10.16.111:5010 ESTABLISHED
    tcp 0 0 10.20.177.1:34033 10.20.177.4:5010 ESTABLISHED
    tcp 0 40 10.20.206.1:34662 10.10.16.110:5010 ESTABLISHED
    tcp 0 40 10.20.206.1:44996 10.10.16.106:5010 ESTABLISHED


    I still can't "ipsisession" to them though. This is from the server on the local LAN so the MPLS network won't play a part here.

    And we're locked down to 100/full.

    As far as QoS goes - we're not doing it. We will in the future but this is a new site and out utilization is nill. We have zero traffic going over this link and I'm not getting any dropped packets.
    David Tessari
    [Avaya]
    What version of ACM are you running?? the distance and nature of MPLS are not optimal for IPSI traffic.. there are improvements in the communication in ACM Rel 4 and the IPSI sanity timer can be extended.. I recommend starting with 6 seconds.. but the max is 14..

    DT
    Inactive Forum User
    We're running 3.1. We had Avaya come in and increase our sanity timers to 8 seconds. This dropped our resets from 4 - 6 per day to 1 - 2 per day. Now I have zero connectivity.

    All Times America/New_York


    Copyright © 2014.'The International Avaya User Group. All Rights Reserved

    All material, files, logos and tradarks within this site are properties of their respective organizations.


    Terms of Service - Privacy Policy - Contact