The traditional approach that rail operators have taken to their communications networks is changing to support new IP video, voice and data applications, as well as improved mobile connectivity and stronger cybersecurity. The advent of the flexible converged network is bringing new challenges, one of which is to turn up the heat on pre-deployment testing. Factory Acceptance Testing (FAT) is no longer enough because, while it adequately covers issues relating to individual components, FAT falls short when it comes to identifying issues that arise when multiple system components come together in a fully integrated system.
The answer is to bring system components together and put them through their paces in a controlled laboratory environment before live deployment. For want of a better name, this approach is known as Integrated FAT (IFAT). But setting up a fully capable laboratory and hiring the necessary experts requires significant upfront investment. It’s easy to imagine the level of expenditure needed won’t pay off, but in fact it’s more than justifiable when the cost-saving benefits are taken into account over the longer term.
The simple reason is that integrated testing improves reliability and drastically reduces network downtime, and every minute of downtime is expensive. That’s all there is to it. Discovering and correcting issues before committing to live traffic is far less costly and disruptive than troubleshooting and correction under the pressures of daily operation. Many organizations have no grasp of the costly ripple effects that network downtime has on their business: lost revenue, lost information, damaged reputations and lost customers.
Leaving aside the rewards in terms of reduced downtime, a laboratory outfitted for IFAT brings with it other valuable benefits. Improved cybersecurity is just one of these. Networks are becoming more enmeshed with IT systems, making them more vulnerable to cyber-attack to a degree that cybersecurity has become a critical issue. For instance, according to the Ponemon Institute’s study, “2017 Cost of Cyber Crime,” the average annualized cost of cybercrime for the transportation industry was $7.36M.
Change control is another area in which lab-based IFAT delivers benefits, in terms of improved reliability and network service quality. Changes equal risk because every change has the potential for unforeseen side effects. For example, imagine you bring up a new circuit between two communication centers and find that application traffic is unexpectedly following an asymmetrical path. Traffic goes out from Comms Center A to Comms Center B on the old circuit, but it comes back on the new one. This is a fairly common scenario—but now there’s a decision to make: Do you try to fix the issue, or back out your change and wait until next month to bring the new circuit into production? What does the change control procedure say? Is there a change control procedure? Will this asymmetrical routing situation even pose a problem?
This is a lot of information to quickly process for an operations tech who most likely does not have a full view of the big picture, and who is running on pizza, Cokes, day-old coffee, and minimal sleep. It is not rare to have an engineer make a small change to fix a routing issue only to cause a major failure. Having a lab facility to duplicate, isolate, make corrections, and develop methods of procedure not only eliminates this risk, but gives your engineers confidence that when they return to the field, everything will go as planned.
Additional valuable benefits of IFAT derive from making full use of the facility as a permanent fixture for ongoing upgrade testing (hardware/software), proofs of concept, staff training, and trouble simulations or disaster recovery drills.
While the transportation industry stands to benefit immensely from advanced networks that can support improved passenger comfort, better real time communication and higher safety standards, the industry needs to go beyond testing components in isolation from one another and embrace deeper and more comprehensive integrated testing in laboratory environments. IFAT offers the best chances of achieving a successful and predictable outcome that avoids costly redesign and troubleshooting during outage operations.