I wrote a list of cyber best practices that appeared in a Radio World ebook in November, “Cybersecurity and Studio Disaster Recovery,” before the current global crisis. RW asked me to revisit and update it given that broadcasters have rushed to find new ways of doing business centered around remote operations and heavy use of the internet.
There are thousands of announcers, account managers, inventory and scheduling staff, programming and music directors, operations directors, engineering managers and other station personnel operating from their homes. How are we handling the IT security and defenses of our operations?
Many of us had to scramble to facilitate multiple work-at-home solutions. Safe practices may have been ignored because the priority was saving businesses or informing our communities.
[Read: Is Your EAS Equipment Secure?]
So now is a good time to assess and reassess. Remember, holes may exist now where they didn’t before, because of emergency actions you took to allow for outside access to systems in your building or transmitter site.
As I wrote in the original version of this article, cybersecurity is a top priority for businesses of all sizes; a lack of readiness and defenses can lead to serious financial and operational consequences. Cyber extortion (ransomware) is big business and is not going away anytime soon. The following questions and thoughts are a place to start in hardening your broadcast organization’s infrastructure and preparing for the worst case.
#1.
Do you have a security-aware culture in your facility? In your organization? Be honest. Knowing that your IT staff or outside contractor installed a new firewall or virus program last year doesn’t mean you are fully prepared. It does not necessarily mean you have a constant security-aware culture that involves regular routines such as:
Backing up crucial data to both a local machine and the cloud and ensuring at least one of the backups is *not* connected to the network source it is backing up.
- Updates and patches are run regularly on all devices such as firewalls, switches, PCs, IOT, etc. We say this all the time but so many facilities do not do it.
- An ongoing awareness and training program for all existing and new employees across all departments. Many attacks arrive via a simple email. Educate everyone about what to look for.
- Antivirus and antimalware software installed on every machine — sounds like Security 101, right? I find machines all the time that are not running both and/or not updated recently with the latest security databases.
- Implemented security restrictions and locked all outside access except where needed. Don’t laugh. I find VPN and Remote Desktop active on machines often, and no one remembers who they were for or what the original purpose was.
- Block all known malicious IP addresses and keep that list constantly updated.
- Keep track of every employee or contractor to whom you gave outside access. Make sure you have a list of their names, systems given access to, and method (VPN, TeamViewer, VNC, public IP, etc.)
This is just a sample listing of key things a security-aware organization should be doing. There are many more. IT trained professionals in cybersecurity know what to do. There are also many excellent sites online with guidelines that dig deeper than we can here.
[Read: IP Security Considerations for Radio Broadcasters]
#2.
Along with #1 above, when was the last time you had a serious sit-down with your IT team, administrator or outside contractor to discuss cybersecurity? How often do you meet? In that meeting, did you know what specific questions to ask? If not, it is time to put together a list of questions. This article can help you get started.
Given the current COVID-19 situation and the fact that you’ve made changes internally to allow for remote access, now is the time for a video conference with the team to inform and discuss any weaknesses. As a team, you can decide what loopholes should be closed now — prioritize any risks should they exist.
#3.
Have you considered hiring a third-party outside security consultant to help with assessing your internal and external systems for their penetrability? Have you asked a trusted security expert to attempt to penetrate your network and systems to ensure you are defended properly?
I know several broadcast-related companies that send phishing emails with fake viruses and ransomware to employees to test their cyber training; see 1(C) above. If the employee clicks on the suspicious attachment, they are provided further training on how to spot these things. The email gateway still ranks as one of the top arrival vectors for attack, so it is critical that everyone have some training on how to spot that one email which can cause you untold hardships.
#4.
Is your network segregated to minimize the damage if something should get through? I often find that networks within the station are combined, on purpose or by mistake. I’ve been in several facilities where they claim their networks are segregated, yet we find that’s not the case.
For example, a PC with a double-NIC (two network cards for separate networks) can be compromised and certain viruses can jump from one network to the other. So the machine that handles traffic but must connect to the automation system — and it is using two network cards — might not be as safe as you thought. Or that one PC that has Remote Desktop on it so someone can get into the network but only though that one “external” machine … well, it may not be the “firewall” you think it is.
There are ways to handle remote access properly and securely. Your trained IT staff or outside security contractor can help you with this.
During the COVID-19 crisis many stations have found themselves needing remote access to their automation playout systems. Normally, as a cyber best practice, these machines are locked down and disconnected from the public internet. If remote capability existed, it was usually through very secure login and VPN methods. I’ve seen many stations in the past month or so that did not have remote access set up allowing their client and server playout machines to be connected to the outside internet. This was done in a hurry and under emergency conditions; some buildings were cleared out almost overnight. If you are one of these facilities, follow #2 above. Make sure management is aware of these temporary weaknesses and address a plan to close the gaps looking forward. You may need this capability in the future, but now you’ll have time to prepare better with more secure access procedures.
#5.
Backup, backup, backup. I mentioned this, but it is so important to preventing disaster that it deserves its own reference. It is imperative that you regularly backup all critical files, and do so to locations that cannot be reached by the virus. There are several cases where ransomware found its way to a network backup and encrypted the very files that were supposed to protect the operation!
Do you backup every 24 hours? Do you maintain backups offsite? (That’s not only a good idea for protection against the virus but also for events such as fire, hurricanes, other things that could keep you from accessing the studio or transmitter location). With backups you can reinstall critical software and data and potentially alleviate the need to pay a ransom. Or it may simply be less costly in time and resources to restore a machine using a recent backup then using a decryption tool. Therefore, very regular backups are crucial.
If for example, you need to restore your music and spot commercial database and audio files quickly, you’ll want that backup to be very recent. Otherwise, you may lose the past several days or weeks of new material — and this could cost the station financially.
I often come across TOCs that supposedly are making backups but are not. The backup tape machine hasn’t worked in who knows how long, the NAS drive is full, the software that runs the backups hasn’t been running for weeks or months, or perhaps the directories selected for backup are not correct.
The takeaway here is that you should ask yourself or your IT administrator for proof that backups are being run, and run often, on a regular recurring basis.
#6.
If you are attacked, do you have the tools in place to quickly detect and determine its origination point within your facility? Do you have the tools (and instructions to staff) in place to isolate the virus or ransomware quickly? Do you use a security event manager? What is your “first 15 minutes” plan?
As mentioned, network segregation is critical in situations where you become infected. If the business network is infected, for example, do you have a way to prevent this attack from spreading to other business networks in your building or within the company (for larger networks or group operators)? Do you have different offices tied together using a WAN/MPLS or other means that might allow the virus to hop over and then start spreading again in an entirely different location?
If you believe a virus is crawling through your network, do you have a plan in place to stop it immediately from moving further along to the next server or PC? Do you know how to kill your network shares immediately? Do you have a plan to yank users and machines from the network in seconds?
What if an attack happens at 3 a.m. on Sunday morning? Do you have the technology or people in place to alert the proper team leaders? And do you have a response go-team on call including holidays?
This is not make-believe or a far-out fantasy. These attacks are happening regularly to small and large operators, and of course, in all industries.
#7.
If your data becomes encrypted, do you have a plan of action filed away so you know what to do? Have you thought about whether you would pay a ransom if presented with such a demand?
There are different schools of thought on whether to pay. Many have paid, and many have not. It is reported by Symantec that only 47% of those who pay the ransom to the bad guys get their data back. It is also claimed by several reputable security firms that if you do pay this time there is a chance you will be hit again because the data kidnappers know you will give in. (Of course, we all know you will be fully protected after the first successful ransom, right?).
Let’s say you don’t pay; better have your recent backups ready to go. Do you have a backup system that provides for restoral easily and quickly? Do you have a go-team put together who will be ready to restore systems and a chain of command to direct team members on what to do and when? (See #6).
If you decide to pay, most ransoms are paid with bitcoin; do you know how to purchase bitcoin? Do you know from where? It can take a few days to obtain bitcoin, depending on how you buy it. Major cities have bitcoin-capable ATMs that can speed this up. The average ransom ranges from a few thousand to much higher. Do you have a source for that kind of money in a hurry should you need it?
Now is the time to think about these things and have a plan written down. If you don’t, you may be scrambling at the last minute while your critical systems are down. That kind of delay can cost you money because your operations are down. If you work with an outside security expert or have such staff internally, and you are not sure what your plans are should you get attacked, ask for one. Do not be unprepared.
On a positive note: Did you know that some ransomware attacks use a software variant that has a free cure? There are free decryption tools out there that might work in your case. Something to check first.
#8.
Some ransomware attacks are widespread. We’ve all heard about them. You’ll see them on TV and on most credible news and IT websites quickly. In some cases, these large-scale attacks are shut down and decrypted within 24 to 48 hours by law enforcement or white hat hackers. If you are affected by one of these large-scale attacks, check with your security provider, consultant, vendor or IT staff to see if there is a fix before paying any ransom.
#9.
If you are in the United States, contact the nearest field office of the FBI or Secret Service and report your ransomware event and request assistance. They may be able to help you. If you are in Europe, go to the Europol website and it will direct you to the local agency in your country. If in Australia, report your event to the Australian Cyber Security Centre. Most countries have a governmental agency that wants to hear from you.
#10.
Ask for help. I say this often. Do not be afraid to ask for help. Whether you are a managing director or engineer and IT director, it is OK to ask for resources to assist you with cybersecurity. You have friends who know things. You have vendors who know things and who have internal resources to assist you with this. There are local IT firms with experts. Consultants. Lots of free advice on the internet. The United States and many other governments provide free information on ransomware, viruses and other forms of malware.
Now, more than ever, we are all coming together to help one another. I’ve seen hundreds of posts online (on the various broadcast-related social platforms) from broadcast engineers, offering advice and asking questions on every imaginable topic related to COVID-19. If you need help with setting up a SIP connection to a mobile phone, there are plenty of people who will help you. Do you need help with remote access to a specific playout system? Just reach out to your vendor or another engineer. Some vendors are offering free versions/use of their remote packages. Every manufacturer and engineer are working together to help one another. I’ve said this before: This is what we do every day; we help stations stay on the air. Even from home!
I walk into too many facilities that are not prepared defensively and that starts at the top. Go back to #1 above. Make sure you have a security-aware culture. Many stations have had to make tough decisions recently on what rules to relax and where the cost/benefit/risk balance lies. This is a decision that is unique to every facility. We are all having to do things differently now than before. Make sure you’ve kept track of what you’ve done so you can go back and close the loopholes. Prepare a list of necessary hardware/software that you can present for approval for things you may need to do this again but with additional security (if needed).
Gary Kline is a broadcast consultant who has held technical positions with several major broadcast organizations, most notably as senior VP of engineering at Cumulus Media. He has provided engineering support and consulting in the United States, Canada, China and several South American countries. He is a past recipient of the Radio World Excellence in Engineering Award.
