Opening a Trouble Ticket for Your Dedicated Circuit

After you have some preliminary information about the trouble with your circuit, you need to call your carrier and open a trouble ticket. It’s at this time that you will realize the benefits of having a technical cut sheet. If you have not made one yet, now is a great time to do so. Check out the Cheat Sheet at the front of this book to set up a cut sheet today.

Going through the basics

If you don’t have a cut sheet, you need to write down the circuit ID for every circuit that is having problems, at a very minimum. Your carrier needs this information to identify your circuit and begin the troubleshooting process. In addition to your circuit ID, your carrier might also ask you the following questions:

• Does your hardware show any alarms? If so, what color are they and where are you seeing these alarms (CSU or multiplexer)? Both the alarms and their location indicate to your carrier what kind of problems your circuit is experiencing. If you have red alarms on your multiplexer, the carrier knows that the circuit is down hard and that your business has no phone service at all. On the other hand, if your CSU is experiencing yellow alarms, your circuit could be intermittently down and bouncing back. If you don’t know this information, you can simply tell your carrier that you haven’t checked yet, but that you still want to open the trouble ticket.

• Have you rebooted your multiplexer/CSU, and if so, how did it affect the problem? You might temporarily resolve some problems by rebooting your hardware. Rebooting enables your hardware to refresh the connection to your carrier and synch up. The only variable you are testing when you reboot is the multiplexer. If the problem clears when you reboot, your hardware is probably the source of your issue. If it persists without any change when you reboot, the source of the problem could be within your carrier.

• What are your hours of operation? In the event that a technician has to be dispatched to your office to fix the problem, your carrier needs to know when someone will be there to let in the technician. If you have an after-hours employee who will be on-site, be sure to provide that employee’s name and phone number (if the system is down hard, provide a cellphone number). You don’t want the technician to arrive and have no idea where to go. If the receptionist or building security has no idea you have a technician on the way, but the technician says, “I was told to ask for Janice Jackson,” he or she is more likely to get through the front door.

• When did you first notice the problem? This information gives your carrier direction in the search. If there was a large outage that hit your area an hour ago, and that is roughly when you noticed the situation, the carrier can easily combine your issue with the overarching trouble ticket (sometimes called a master trouble ticket). Of course, your problem may not be related to a larger issue, but offering a time frame does give your carrier some indication of events that occurred that may have caused your issue.

• Who is the site contact? Your carrier needs to know who to call for updates. Please provide a direct phone number, a cellphone number, and a secondary means of contact if your contact plans on being unavailable at any time during the day. It’s painful to pick up a voicemail from your cellphone after you have lost four hours because you ran into a meeting, got busy, and didn’t have time to listen to it until the end of the day.

• Is the circuit released for intrusive testing? Intrusive testing is the most direct way your carrier can investigate an issue with a dedicated circuit. The main downside of intrusive testing is that your circuit will be completely down while your carrier takes over every channel. Of course, if the system is already down, you have nothing to lose. On the other hand, if your circuit is active when you release it for intrusive testing, any active calls will be disconnected when the carrier initiates the test. See the section, later in this chapter, “Step 2: Intrusively testing: Looping the CSU” for more information about timing your test.

Letting your channels be your guide

When you open a trouble ticket, it’s always helpful to ask the carrier for a circuit snapshot. A snapshot of the circuit is the disposition of each channel on your circuit. Even if you think you have a DS-1 level issue, it’s always a good practice to ask for this information. You might believe that your entire circuit is being affected, but a snapshot could reveal that only half of the channels are impaired. With basic facts from the PMs and a circuit snapshot, you can begin setting realistic expectations for resolution of your service. Additionally, you can isolate individual DS-0 issues to the specific channels experiencing the problem. To solve your problem, you might be able to reboot your phone system to bring the channels back into service, or have your phone system deselect them as available lines. The key is using every available bit of information your circuit provides. Taking in the disposition of your individual DS-0 channels on your DS-1 circuit is a quick and easy way to glean what is happening to your circuit.

Remembering the first rule of troubleshooting

 Remember  The first rule of troubleshooting is to keep an open mind. Suspecting that a problem can be attributed to a particular source is fine, but don’t be so sure you’re right that you lose perspective. All the circuit states I describe in the following sections do suggest a likely source for the problem you’re experiencing with your dedicated circuit. Most of the time, for example, a channel in RMB (remote made busy) state is having hardware troubles. However, there is that tiny chance that a glitch within your carrier is causing it to send the channel a message, or triggering the channel to sit in RMB. Alternative scenarios are always possible, not just for channels in an RMB state, but for all the circuit states listed in this chapter. Use the information to guide your troubleshooting, but don’t be so hung up on a single right answer that you dis-count the possibility that something else is causing the issue.

Idle: IDL

Your channels are in an idle state when they have access to your carrier’s network, but don’t have an active call. This is the state you expect your channels to be in when everything is working fine and the circuit is waiting to process a call.

 Remember  The circuit must be considered idle by both your carrier’s technology and your own phone system in order for a call to come through on the line. If, according to your carrier, the channels are idle, but your hardware is unable to seize the channels for some reason, one end of the circuit isn’t speaking to the other. Open a trouble ticket. The problem might be between your phone system and your multiplexer, or your multiplexer and your CSU. If your carrier sees the circuit as active and you can’t seize a channel, there must be a piece of hardware interacting with your carrier to make the carrier think everything is fine.

Call processing busy: CPB

Call processing busy or CPB is the healthy state of a channel with an active call on it. Under normal circumstances, your active circuit has several channels in the CPB state, indicating active calls on the specific DS-0s. The rest of the channels are idle, waiting to accept a call. A circuit with channels in CPB is obviously connected to the carrier, because only a call that is connected to the network can establish the channel in this state.

Remote made busy: RMB

Remote made busy or RMB is a common disposition for a channel if you’re dealing with hardware issues. The carrier may see a DS-0 in this state when your hardware independently locks out the channel. Channels in an RMB state can sometimes be corrected if your carrier takes down the specific channel and resets it (this process is commonly called bouncing a channel). If your carrier can’t bring the DS-0 back into service by bouncing it, the only way to restore the channel is for you to reboot the hardware. If this condition is chronic, contact your hardware vendor to test the hardware. It’s better to find the problem when it’s minor, than to let it go until the point at which the channel won’t come back to life after you reboot.

D channel made busy: DMB

The D channel made busy state is used only with ISDN circuits. Only ISDN circuits have D channels designated for signaling, and that is why non-ISDN circuits can’t have channels in this state. When the D channel made busy state appears on an ISDN circuit, the entire circuit is down. The D channel handles all the logistics of call setup and teardown on an ISDN circuit, and without the overhead, the circuit can’t function.

 Tip  This condition can be the result of your hardware having issues (see the section on RMB, earlier in this chapter), your carrier having issues, or an ISDN protocol mismatch.

Installation made busy: IMB

Installation made busy, or IMB, is a protected state that is usually imposed by your carrier on an entire circuit to prevent it from generating alarms in the carrier’s network. For example, if you unplug your hardware from the circuit, your carrier immediately sees alarms at its end of the span. The technician who is listening to the piercing screech of the alarm doesn’t have your contact name and phone number, so he or she does the easiest thing to end the maddening alarm. Usually, the easiest thing to do is to busy out the circuit, or place the circuit into the IMB state. The bad news is that when you plug your hardware back in, you won’t be able to use it because the carrier has been busied it out.

 Remember  Every carrier has its own procedure for placing a circuit into the IMB state. The process might be manual, with a policy to wait until the circuit has been in alarm mode for eight or ten hours before busying out the circuit. Sometimes, however, the process is automated; you may have as little as an hour before the carrier automatically busies out your system.

Carrier failure: CFL

Carrier failure is like IMB, because it generally affects your entire circuit, and not just a few channels. The bad news about a circuit in carrier failure is that this state commonly indicates a substantial problem that needs to be addressed pronto by your carrier. If your CSU fails, or if someone accidentally cuts through the wiring of your circuit with a backhoe, or if your carrier’s switch sustains a direct lightning strike, your circuit is in carrier failure.

Avoiding permanent IMB status

 Tip  To prevent your circuit from spending more time in IMB than it needs to, open an information-only trouble ticket with your carrier before unplugging your hardware for any reason. This type of trouble ticket alerts the carrier to the fact that you will be removing your hardware from the circuit and gives you a trouble ticket number to reference when you call into have the circuit taken out of IMB.

If you don’t want to go through this effort, simply place a loopback plug in the T-1 jack of the circuit from which you’re disconnecting. This simple action tricks your carrier into thinking there is hardware connected on your end of the circuit for as long as the loopback plug is in place. This isn’t an option if you have ISDN circuits, because your carrier needs to chat with your D channels to ensure that the hardware is active. The ISDN protocol is generally intelligent enough to know that it is speaking to itself and eventually places the circuit in alarm.

Removing a circuit from IMB takes about five minutes, after you reach a technician with the required credentials to make the change. The greatest potential delay comes from wading through the customer service department and waiting for the magic technician with the access to the switch to call you back.

 Remember  Carrier failure indicates a more serious issue and generally takes several hours to diagnose. After the technician finds the source of the problem, it can be another 4 to 24 hours before your circuit is back in service. Keep this in mind when you set expectations. (In other words, it’s time to have a chat with the head of your telemarketing department.)