Note: this blog is a mirror of my HP Labs Blog, on the same topic, accessible at: http://h30507.www3.hp.com/t5/Research-on-Security-and/bg-p/163

Friday, September 6, 2013

Big Data for Security @ HP Labs: Key Milestone Achieved


In the Big Data for Security R&D project, at HP Labs, we achieved an important milestone. We delivered our first, fully working prototype (and related demonstrator) illustrating how  it is possible to analyse Big Security data to identify potential (new) security threats and issues of relevance to organisations.

We focused, as a case study, on DNS events: DNS logs are usually huge, due to the very large amount of DNS queries (and replies) performed per second. As a consequence, companies usually fail in logging this type of information or they restrict the collection/retention to very small time periods. On the other hand, DNS Infrastructure is critical and can be used to launch attacks and/or for criminal intents.

Hence, being able to analyse DNS logs (potentially in conjunction with other logs) is key to identify attacks and misbehaviours.

Our demonstrator analyses DNS logs (currently only DNS queries, in the near future also DNS replies) and provides insights about potential security threats and issues. This is achieved via Historical (Security) Analytics and Visualization capabilities developed at HP Labs.

We fully leverage current HP Software and Security (HAVEn) solutions, Including HP ArcSight Logger, HP ArcSight ESM, HP Vertica and HP RepSM.

In the coming months we aim to:

·         Refine this solution by including advanced anomaly detection functions, trend analysis and machine learning, coupled with compelling visualization;

·         Process a wide range of data types, beyond DNS logs (e.g. web proxy logs, IPS logs, vulnerability scanning logs, user access logs, etc.)  along with related analytics;

·         Process and analyse unstructured data, by leveraging HP Autonomy;

·         Leverage distributed analytics solutions (including Hadoop) and advanced statistical tools (e.g. R).   

This is work in progress. We are currently showcasing this solution to HP customers and partners to gather additional requirements and feedback. More to come in the coming months.


--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

 

On cyber attacks leveraging DNS: where are the numbers?


DNS is a critical infrastructure: it is often leveraged to launch attacks and/or for criminal intents. Recent news further demonstrates that the impact and consequences of such cyber attacks could be extremely serious.

I am looking for public statistics, reports, analysis, etc. that quantify the amount of cyber attacks  that directly or indirectly exploit DNS infrastructure. Any help (links, documents, etc.) would be really appreciated.

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

My Tweets of the Week (02-07 September 2013)


My tweets of the week (02-07 September 2013) at https://twitter.com/MCasassaMont:

·         Aussie startup Packetloop turns big data APT forensics into Arbor Networks success - http://www.cso.com.au/article/525707/aussie_startup_packetloop_turns_big_data_apt_forensics_into_arbor_networks_success/  #in

·         How Big Data Is Transforming The Mobile Industry - http://www.businessinsider.com/local-mobile-makes-big-data-real-2013-9  #in


·         Looking for public statistics about cyber attacks that exploit/leverage DNS infrastructure and related capabilities #in

·         3 Lessons CMOs Take Away From IT's Flawed Approach to Big Data - http://www.cio.com/article/739057/3_Lessons_CMOs_Take_Away_From_IT_s_Flawed_Approach_to_Big_Data  #in


·         Preparing for HP customer meeting. Presenting and demonstrating our R&D work on Situational Awareness and Big Data for Security #in

·         Exploring the Big Data Stack - http://www.datamation.com/data-center/exploring-the-big-data-stack.html 


·         10 hard-earned lessons of a lifetime in IT - http://www.infoworld.com/slideshow/116989/10-hard-earned-lessons-of-lifetime-in-it-225919  #in


·         A Closer Look into the Future Big Data Ecosystem [INFOGRAPHIC] - http://smartdatacollective.com/bigdatastartups/141626/closer-look-future-big-data-ecosystem-infographic  #in

·         Tor Use Uptick - http://isc.sans.edu #in

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---