IT Infrastructure Rebuild Behind Dish’s Transformation
October 17, 2012, Storage Network World, Santa Clara, C—Mike McClaskey from Dish Network described the efforts and results of a major IT infrastructure rebuild on the company’s business. The IT overhaul affected most of the departments in the company, so a lot of planning and trials had to be done before the cutover.
Dish planned to make a major infrastructure change in a short time. To drive a big project like this, IT has to enable the transformation of the whole business. Dish has over 14 M customers and brings in about $14 B per year. Now the company is in a time of change and is not just a satellite company, but a full service communications provider. Moving from satellite TV to a quad-play participant required changes in systems and apps.
The company acquired cell phone spectrum, and needed to transform the business. They made investments in many new areas, including billing and customer service. The four cornerstones for their change were: IT continues to grow bigger and more powerful. The multi-play environment needs billing for pay TV, wireless, telephony, etc. as well as a dispatch system, new phone systems for customer contact, and customer care, etc. Their decisions were based on the pillars of technology, vendor choice, hosting, and cloud services.
Second, they focused on strategic issues such as cloud versus owned infrastructure, and carefully considered the alternatives. Third, the network infrastructure needed significant investments to bring the WAN, LAN, and SAN functions up to speed and able to met the requirements for operations. And fourth, they had to get the new system into production within a short cut-over period. They accomplished all these requirements in under two years and less than $100 M investment.
The keys to their success were: to develop and communicate a compelling vision to everyone. The billing systems needed to expand to cover mobile, TV, wireless, etc., so they had to invest in infrastructure even before they bought the wireless licenses. The billing system expanded to include workforce management, call centers, and other functions, so there was a snowball effect that made everything grow. To combat the creeping requirements, they controlled everything by refining the focus.
They worked to limit the scope of the project. All of the investments are able to scale and the justifications for the investments had to make a total business case. The business cases had to recognize that some times a change will save in other areas, even though it might not have a good ROI for that specific department.
Second, they built teams and worked to get the right people for the jobs. The hiring practices called for getting people with high energy, intelligence as determined through standard testing, and a need for achievement. They instilled a sense of pride, adventure, and winning into all the teams, but also needed leaders and facilitators to encourage more collaboration. One necessary requirement was a need to be calm under pressure, as this was in a 24/7 environment. They didn’t hire any consultants and only a few contractors.
Third, was to get the right partners. They developed exhaustive selection criteria and preferred leading- over bleeding-edge vendors. They did not exclude small companies as vendors, because a good solution does not always come from the big, established companies. For example, their hosting strategy called for shared risk-reward. Partners were involved in key technological decisions, like the billing systems by the billing provider, and IVR systems by the IVR provider. Other functions were from internal groups.
Their data center of the future is located in Cheyenne, Wyoming. It has a dedicated, private dense wavelength division multiplexing fiber ring based on existing dark fiber. It has over 1.6 TB capacity on quad redundancy. This infrastructure allows them to self-host their 1-800 lines and transfer the calls to carriers on demand.
And finally, disciplined execution. They structured their teams like building ladders, so everything could scale up as needed. They had a “take the hill” mentality and made all the workers know the timeframes, and goals for the projects. They de-stacked risk by practicing all of the tasks many times outside of production, and set up everything for staged turn-on. This process allowed them to take a changeover from 2 weeks to 8 hours. They performed 11 tests in 6 months and, at the end, could complete a changeover in under 6.5 hours.
The investments in extra hardware allowed them the ability to practice and test the systems off-line. They set up “war rooms” and created standard software release processes. And possibly one of the most important functions was to celebrate successes.
