In the Big Data for Security R&D project, at HP Labs, we
achieved an important milestone. We delivered our first, fully working
prototype (and related demonstrator) illustrating how it is possible to analyse Big Security data
to identify potential (new) security threats and issues of relevance to
organisations.
We focused, as a case study, on DNS events: DNS logs are
usually huge, due to the very large amount of DNS queries (and replies)
performed per second. As a consequence, companies usually fail in logging this
type of information or they restrict the collection/retention to very small
time periods. On the other hand, DNS Infrastructure is critical and can be used
to launch attacks and/or for criminal intents.
Hence, being able to analyse DNS logs (potentially in
conjunction with other logs) is key to identify attacks and misbehaviours.
Our demonstrator analyses DNS logs (currently only DNS
queries, in the near future also DNS replies) and provides insights about
potential security threats and issues. This is achieved via Historical (Security)
Analytics and Visualization capabilities developed at HP Labs.
We fully leverage current HP Software and Security (HAVEn) solutions,
Including HP ArcSight Logger, HP ArcSight ESM, HP Vertica and HP RepSM.
In the coming months we aim to:
·
Refine
this solution by including advanced anomaly detection functions, trend analysis
and machine learning, coupled with compelling visualization;
·
Process
a wide range of data types, beyond DNS logs (e.g. web proxy logs, IPS logs,
vulnerability scanning logs, user access logs, etc.) along with related analytics;
·
Process
and analyse unstructured data, by leveraging HP Autonomy;
·
Leverage
distributed analytics solutions (including Hadoop) and advanced statistical
tools (e.g. R).
This is work in progress. We are currently showcasing this
solution to HP customers and partners to gather additional requirements and
feedback. More to come in the coming months.
