VoIP Response to ITC - 11/13/02

There were some questions raised at the ITC meeting concerning the proposal from NETS and NCAB to fully deploy Voice over IP and related technology throughout the NCAR/UCAR campuses. Below we hope to address those questions.

The approximate cost of full VoIP deployment across UCAR per the linked plan is: $1.2M
- It is difficult to give exact costs until the space planning is complete as far as how many buildings and campuses we are supporting, but this is a reasonable estimate
- We plan to spread these costs over 2 fiscal years
- Approximately $300,000 over two fiscal years will be handled out of general NETS equipment funds and bond equipment funds for new campus build-outs
- Approximately$875,000 will be handled by existing NETS M&S funds over two fiscal years and bond infrastructure funds for new campus build-outs
- Approximately $25,000 will be handled by security funds
Security and privacy issues:
- What is stored on the voicemail servers?
  - Voicemail messages are stored on a voicemail server as they are today. The new voicemail servers are highly
    redundant; dual power supplies, RAID disk array. Messages on the new system are backed-up, the old system only backed up the database.
- Does anyone see messages, etc.?
  - No one sees these messages
- How does the VoIP system compare to the current phone and voice messaging systems in this regard?
  - The voice traffic travels on its own protected VLAN, very similar to the current dedicated circuit emulation being used today. This traffic is very secure, short of someone breaking into a switch internal to our network and sniffing packets.
  - It is quite possible to encrypt voice traffic, in general, although it is currently not a feature on our traditional or VoIP systems. Cisco is working on providing QoS for Virtual Private Network (VPN) traffic. Once this feature is complete, if we ever exchange VoIP traffic with other sites, this should allow us to encrypt that traffic in the Wide Areas Network (WAN).
Do the phones go down when the network goes down?
- The answer is, it depends:
  - If we lose a single switch, the phones connected to that switch go down - same as today but fewer phones are affected in new system since the phones are spread to more switches so fewer phones per switch.
    - There currently are five PBX switches at the ML, FL, FL4, PS and Jeffco campuses today handling approximately 1800 phones
    - There are approximately 25 Ethernet switches today that would handle that same 1800 phones
    - There is no outage of a single Ethernet switch on the VoIP system that would be as catastrophic as losing the ML or FL PBX switches today
  - If we lose all links to a site such as FL, the phones will be isolated with local access only - same as today. The new system
    will be less vulnerable to this since the new system uses our network links, which have multiple paths. Our current voice traffic
    travels along a single path to each site.
  - Should a disaster occur and a site becomes isolated, contingencies include cell phones (we are deploying a Nextel cell system at the ML so that cell phones will work there) and if a site is operational, just cut off, there are direct Qwest lines at each site to dial out using 9 plus a 10 digit number to NCAR or elsewhere - this is the same set of contingencies that exist with the current PBX system.
  - A rough summary of the outages we have experienced in the last 5 years on the PBX system is:
    - 9 link problems between PBXs which would have isolated sites
    - 11 PBX outages - anywhere from system restarts to UPSA & RAP UPS outages and PS disk drive failure without good tape backup
    - 7 voicemail outages - only one with lost messages
Who was involved in testing?
- The list of testers was provided in the original report to the ITC, but here it is again:
  - The NCAB members that tested the VoIP system are: Mark Bradford/JOSS, Tim
    Fredrick/ACD, Tres Hofmeister/RAP, Terry Murray/DIR/ISS, Bunny Pfau/HAO, Mark Moore/CGD, Brandon Slaten/ATD, Pat Waukau/MMM, Greg Woods/CISL
  - All 25 NETS staff members tested the VoIP system.
Risk and risk mitigation plan, from both technical and business perspectives.
- As with all network technology, the technical risks are hardware, software, and staffing. We have attempted to minimize technical risks as follows:
  - The hardware being deployed is highly redundant: dual power supplies, dual supervisor engines, dedicated UPS per switch, back-up call managers, back-up voicemail servers, back-up network paths.
  - Software is under maintenance and is fully backed-up on a daily basis.
  - Previously, Telecommunications relied heavily on off-site vendor support since there were limited staff, now more NETS staff have been trained and response is quicker.
- I am not aware of any business risks that differ between traditional and VoIP, but here is a summary of areas where we have addressed business risks:
  - This is a stable and standardized technology which allows us to use multiple vendors which is not true in traditional voice systems and should minimize our business risks.
  - Cisco is a large, stable networking company committed to VoIP.
  - The distributed nature of VoIP allows for upgrades more quickly to add features versus a 10 year life of a traditional system. This mitigates business risk of a long term commitment to a single and static technology.
  - The VoIP system would be simpler to deploy in a business continuity emergency since both the data and voice systems use the same infrastructure.
Was there a user satisfaction survey?
- There was a detailed discussion with NCAB and NETS testers and responses are in NCAB minutes. This was not a formal written survey.
- There is ongoing input from NCAB and NETS testers to the VoIP deployment team to address any issues.
- The design and testing of VoIP has been done for over a year with constant input from Cisco, Westnet, NCAB and NETS.
There is clearly a networking strategy at UCAR, is there a telecommunications strategy? If so, is it integrated with the networking strategy?
- Although Telecommunications has a long history of being well-managed and well-maintained at NCAR/UCAR, there has never been a formal Telecommunications strategic document for UCAR to my knowledge. Since Telecommunications has been integrated with NETS, the Telecommunications strategy is in the process of being integrated with the larger NETS strategy. Basil Irwin is currently working on the updated NETS Strategic Plan which will integrate Telecommunications strategy as well. This will include the recommendation for VoIP deployment as part of a larger networking strategy as has been discussed in detail within NETS and NCAB.
- NCAB has become the advisory body for telecommunications as well as networking and has been integral to the VoIP research, testing, and deployment process.
Long distance and site interconnection are potential future uses of VoIP.
- To clarify, UCAR campus site interconnection will be accomplished as part of the VoIP deployment.
- There are definite future possibilities of Metropolitan Area Network (MAN) and WAN VoIP networks as VoIP is deployed more extensively. In fact, we have already discussed options of using BRAN with CU-Boulder, the City, and the Department of Commerce labs for VoIP deployment within Boulder. This would minimize the number of lines required for outside calls which cost us monthly fees and keep traffic on free BRAN bandwidth. Feasibly, as we acquire dark fiber that we own and control, this paradigm of using the network for voice traffic could expand to the FRGP and potentially National LightRail partners which would further reduce long distance charges.
Is the VoIP system standards-based?
- Yes, it is fully standards based.
Cost/Benefit Analysis:

Cost of VoIP	Benefit	Cost of Traditional	Benefit
$937 /port with phone $128 /user voicemail		$1500 /port with phone $250 /user voicemail
	Employees with offices in multiple buildings could have same phone number and single voice message box		The main benefit of plain old telephone service (POTS) over VoIP that is usually presented is the same as the flip side cost savings of VoIP: totally separate systems and cabling infrastructure. If you cut just the data cable, your PBX phone (potentially) still works (though as noted elsewhere in the report, that depends on the physical layer design quite heavily and as we move to shared fiber networks will become less and less true as an advantage). However, you've paid for two systems and cables so the cost of maintaining both systems goes up. This is really a perceived comfort level because it is existing and familiar technology and physical design.
	A single cabling system to provide both network and telephone service
	Simplification of adds/moves/changes for telephones
	Users pick up their phones and move to their new office
	Ease of expansion to the system - scales well
	Ease of adding/upgrading features
	Standardization of equipment for both data and voice facilitates training and increases breadth of expertise
	Network test equipment can be used for both data and voice
	Redundant links between network devices have been added to the network to increase reliability of both data and voice
	The ability of VoIP to provide limited re-routing of inbound calls during a link outage for emergency purposes, which is not available on the current system
	Advanced re-dial and directory features
	Potential cost saving of local phone lines (fewer needed)
	Potential cost savings of long distance calls to other VoIP centers
	Single equipment location for both data and voice equipment
	Less spare equipment needed since we are sparing one network system instead of a network and PBX system
	System setup time is the same or less then traditional telephone systems
	Single system deployment
	Potential for improved E911 services, including detailed phone locator
	Physical security is enhanced because phones can be tracked
	A number of our partner universities and institutions are also in the process of deploying VoIP for the same reasons as outlined above, and we can and have leveraged their expertise.