Securing Big Data at SNIA
September 19, 2013, SNIA Analytics and Big Data Summit, Santa Clara, CA—Scott Burgess from EMC discussed the issues related to securing big data. The requirements for security are much more complex than for older data warehouses.
The areas as of vulnerability are related to the big data technologies like Hadoop and Hadoop Map/Reduce. The big data are not just another data warehouse where information management is a part of the business integration and includes data management, analytics, reporting in a closed-loop processing operation. Data sources go through the systems through some delivery mechanism to the users.
Now, the architectures are changing to address the challenges of big data. Data are stored and queried in a Hadoop distributed file system (HDFS) and most of those data are unstructured and include social feeds, video, documents, XML, etc. The data are chopped up and stored on the HDFS which scatters those data in numerous nodes for built-in fault tolerance.
The HDFS has one master/name node and many slave/data nodes. The name node stores metadata while the data nodes store the actual data. All of these data are stored on commodity x86 severs, and each node offers local storage and compute. Queries are submitted to the master node which “maps” the process to slave data nodes. The sub-jobs are executed in parallel against each node’s local dataset. The slaves complete and return results to the master which “reduces” the aggregated, assembled results and sends them to the client.
The issues are data at rest, host node access through good operational hygiene and strong authentication schemes, node and API authentication, and transport encryption. Control point management, logging, and SEIM complete the picture. Some solutions use the nature of big data to secure the big data. SIEM and SOC integration and control point automation can all work together to secure the entire data set.
Live intelligence allow the distributed collections to traverse the real-time and warehouse paths while flexible integration through APIs allows the analytics to report and alert events that require investigation. Malware analytics and administration handle the complex event processing to correlate intrusions with other information. The metadata tagging is important for both user and for security. The whole system enforces incident management that is graded through asset criticality and compliance to various requirements.
It is important to choose solutions and providers carefully. The Hadoop ecosystem has many participants and those with the largest number of partners are more likely to have the higher-quality solutions relative to security issues.
