Note: this blog is a mirror of my HP Labs Blog, on the same topic, accessible at: http://h30507.www3.hp.com/t5/Research-on-Security-and/bg-p/163

Sunday, July 28, 2013

Big Data for Security: On Using DNS Logs for Security Threat Detection


I am particularly interested in the area resulting from the intersection of the following topics: big data for security, big data analytics, distributed programming and data analysis solutions, security and cloud.

 

In particular I am interested in public case studies, business cases and trials involving the usage of (large amounts of) DNS data to detect new security threats and issues.

 

Here are some key related work and approaches:

·         EXPOSURE: Finding Malicious Domains Using Passive DNS Analysis;

·         Large scale DNS analysis





·         DNS-based Detection of Scanning Worms in an Enterprise Network



  

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

My Updated Work and Personal Web Sites


I recently updated my personal and HP Labs web sites with news, deliverables and achievements, in the space of Security, Cloud and Big Data:

·         My HP Labs Web Page

·         My Personal Web Page

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

My Tweets of the Week (22-26 July 2013)


My tweets of the week (22-26 July 2013) at https://twitter.com/MCasassaMont:

 


·         Three different roads to the 3-nanometer chip - theregister.co.uk/2013/07/25/pro… #in

·         With big data comes big responsibility - ft.com/cms/s/0/1c3e27… #in

·         “Big Data” Is Not “Big Data” Unless It Gives You Actionable Insight - searchengineland.com/big-data-is-no… #in


·         Researchers spot new breed of infected Android apps in the wild - infoworld.com/t/android/rese… #in

·         Graph analysis will make big data even bigger - infoworld.com/d/big-data/gra… #in

·         CFOs Ignore Big Data at Their Peril - online.wsj.com/article/SB1000… #in

·         Competing businesses encouraged to share incident data as the attackers do - scmagazineuk.com/competing-busi… #in

·         FTSE 350 companies demonstrate very poor security manners - scmagazineuk.com/ftse-350-compa… #in

·         Stop 80 percent of malicious attacks now - infoworld.com/d/security/sto… #in

·         SDN 101: Software-defined networking explained in 10 easy steps - infoworld.com/slideshow/1117… #in

·         Big Data Security Analytics: It Takes a Village - networkworld.com/community/node… #in

·         Software employment rises 45% in 10 years, as angst in engineering grows - computerworld.com/s/article/9240… #in

·         Happy birthday, OpenStack! Now change - networkworld.com/news/2013/0722… #in

·         True tales of (mostly) white-hat hacking - infoworld.com/d/security/tru… #in

·         Five Roles You Need on Your Big Data Team - blogs.hbr.org/cs/2013/07/fiv… #in

·         SIM card DES flaw could affect up to 500 million users - scmagazineuk.com #in

·         Are we in an enterprise startup bubble? - infoworld.com/t/startups/are… #in

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

 

Friday, July 19, 2013

On Big Data for Security


I am currently focusing my R&D work in the space of “Big Data for Security”.

This is a fascinating area and, currently, a green field.

 

How to effectively leverage huge amount of collected IT information (ranging from IT logs to application and service information as well as external intelligence)  to identify new security threats, issues and provide valuable information to organisations to mitigate current and foreseeable risks?

 

HP already has core assets in the security and “Big Data” space: HP ArcSight suite (SIEM solution for event logging, storage and correlation); HP Vertica (highly parallelised, columnar database solution for storage and analytics of structured big data) and HP Autonomy (storage, indexing and retrieval of massive amount of unstructured data).

 

I am currently exploring how these capabilities could be fully leveraged in the context of big data for security, in particular in a few security verticals and types of critical security data. In addition, I am interested in exploring how the massive amount of required computation and analytics can be performed by adopting innovative solutions in the cloud (private and hybrid cloud).

 

I am looking for public use cases, case studies and requirements in this space, in particular for analytics based on big security data and anecdotes on how “big data” has been helping to address security issues.

 

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

 

HP Labs Research


HP Labs has updated its Research web page, providing an overview of its 5 key research focus areas. More information is available here.

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

On HP Moonshot Server Solutions


Recently HP launched the HP Moonshoot server solutions.

An interesting video titled “Meet the Innovators behind HP Moonshoot” is available online, here.

 

--- Posted by Marco Casassa Mont (here and here)  ---

--- NOTE:  use this mirror blog if you prefer posting on an external blog site  ---

--- NOTE:  my original HP blog can be found here  ---

Wednesday, January 9, 2013

More on Safe Information Sharing in the Cloud

With the adoption of services in the Cloud, organisations inevitably lose control on their IT and might lack the critical information required to assess a variety of (business, performance and security) risks.


Traditional approaches based on SLAs and contractual agreements only partially address the above issues, as they provide only a “predefined” and static “view” of the situation which does not cope well against fully dynamic, ever changing IT operations and threat landscapes.

In this context, enabling more dynamic, controlled information sharing in the Cloud is key to improve situational awareness and address the above issues. This involves dealing with tension points between information sharers and sharees (about what to share, why to share, how to control information flows, etc.) along with trust and assurance issues.
More R&D is required in this area, in particular on how to provide safe information sharing and the relevant controls on the information flows.
At HP Labs, Cloud and Security Lab (CSL), we work in this space: we aim at shaping the vision and providing concrete solutions to be used in the market.
In previous blogs of mine, I provided an overview of our vision and related demonstrators we developed to convey it, in the space of Situational Awareness and Information Sharing in the Cloud, in particular in the context of Disaggregated IT.

I also briefly discussed the R&D work we do to provide better predictive analytics based on collected and shared data, in particular in the area of strategic security risk assessment (see our work on SILAS - Security Intelligence-as-a-Service).

I am looking for additional, concrete examples and case studies illustrating how current cloud adopters cope with situational awareness and assessment of the involved IT operation and business risks – including pros and cons or current approaches. Your input in terms of requirements, scenarios and feedback is welcome.

--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---

On Policy Decision Support for Big Data

When dealing with big data (inclusive of hybrid and unstructured one), it is very hard to understand the implications and impact of defining (security, business, sharing, privacy, etc.) policies on this data.
Which data is actually affected by the policies? Are these policies comprehensive? Are there corner cases that are not covered? Further complexity is introduced by the fact that analytics can be performed on big data, whose outcomes and implications are unknown at priori, as well.
Decision support tools are required to help policy makers to explore the implications of defining policies on big data and related analytics. In my view, these tools must provide synthetic visualization of big data as well as real-time feedback on the implications of defining specific policies and related constraints.
I am looking for:

- Tools providing synthetic, visualization of big data as well as potential analytics

- Public documentation, approaches, case studies, solutions, etc. describing how businesses currently cope with the consequences of defining (security, business, privacy, sharing, etc.) policies on big data



--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---

Thursday, December 13, 2012

More on SILAS: Security Intelligence-as-a-Service

In a previous blog post of mine I introduced our HPL/HP work on the Security Intelligence-as-a-Service (SILAS) solution and the fact we achieved an important milestone, in collaboration with HP business groups: a full working implementation is available.


Thanks for your questions. I am providing some additional details. The SILAS solution can now be showcases to HP customers and (potential) business partners.

As previously mentioned, SILAS consists, at the very base, of an Analytics Technology that provides: statistical analysis of data; predictions based on simulations.

There is currently a major gap in organizations’ security lifecycle management processes. On the one hand, organizations carry out strategic, long-term risk assessment activities - at the business level - to identify threats and mitigate them with suitable policies and controls. This involves periodic re-assessment of their security investments. On the other hand, they heavily invest in monitoring and Security Information and Event Management solutions (SIEM - e.g. HP ArcSight) to collect information from their IT infrastructure, for compliance and governance purposes. However information gathered at this level is seldom leveraged for higher-level strategic security risk assessment, except by means of expensive and manual processes. It is primarily used at the IT Operational levels. There is increasing demand for better integration and simplification of these processes in order to maximize investments and improve the overall risk assessment.

This gap is even more evident in the context of managed services and/or disaggregated IT in the Cloud, where the organisation further loses control on their IT along with related information flows. SILAS aims at addressing this gap.

A typical scenario (where SILAS can be deployed to add value) consists of a multitenant Security Operation Center (SOC), as shown in the following picture:





In this scenario the SOC manages incidents and IT operation issues for multiple customers. SILAS calculates and provides a wide variety of strategic metrics:

• customer metrics, reflecting the effectiveness of their processes (e.g. vulnerability and threat management - VTM, identity and access management - IAM, etc.), based on the data they shared with the SOC; metrics related to external threat environments (e.g. derived from information collected from HP ArchSight, HP TippingPoint, DV Labs, OSVDB, etc.);

• metrics providing an assessment of SOC processes, e.g. how effectively they identify incidents, close alerts, deal with false positives;

• what-if analysis and predictive metrics.

SILAS is meant to:

• provide estimation of strategic (security, risk and business) metrics to decision makers and customers, in multi-tenancy, multi-customer contexts, such as Security Operation Centers and Cloud Operation Centers

• use these metrics to enable predictive and what-if analysis, by leveraging the HP/HPL Security Analytics Solution (based on modelling and simulation techniques)

• provide customers with strategic reports - based on processed metrics and prediction - to illustrate historical trends and benchmarks

• leverage Cloud infrastructure for data processing and metric estimations

The following picture illustrates the SILAS core capabilities and high-level architecture:



SILAS is not meant to be a reactive, real-time analytic solution. It leverages existing solutions such as HP ArchSight, HP TippingPoint/ThreatLinq, OSVDB, etc. to gather the relevant data. As unique differentiation, it provides longer-term estimates of critical metrics and uses them to make predictions. It provides decision support capabilities to key stakeholders (risk management teams, customers, etc). As such it nicely complements current HP SW offerings.

We are currently trialling this solution in collaboration with HP business groups.

A few screenshots of a public version of SILAS (we use for demonstration purposes) follow:




Figure 1: SILAS main dashboard. Links to various metric processing, prediction and reporting capabilities




Figure 2: SILAS metric estimation. Example of estimation of "patch take-up curve" metric estimation (i.e. how quickly an organisation patches its systems against a vulnerability), over a period of time, calculated on data collected from HP ArcSight




Figure 3: SILAS predictions and "what-if" analysis. Example of prediction to vulnerability "risk exposure", calculated with HP/HPL Security Analytics models and related simulations. Models are instantiated with previously calculated SILAS metrics, e.g. the "patch take-up curve" metric.




Figure 4: SILAS Report. Example of customer report illustrating, for a given time period, the "patch take-up curve" metric and compareing it against an anonymised version of the same metrics (in the same time period)/benchmark,  calculated by using information collected from other customers (in a multi-tenant SOC).




Figure 5: SILAS Report. Another example of customer report showing the outcomes of various "what-if" analysis, calculated with HP/HPL Security Analytics models and related simulations. Models are are instantiated with both previously calculated SILAS metrics, e.g. the "patch take-up curve" metric and the various "what-if" assumption to be explored (e.g. using specific IT security controls).




Figure 6: SILAS Report. Another example of customer report showing the historical trends of some relevant SOC process metrics indicating how effectively a SOC handles customer's incidents (e.g. in terms of time to close an alert, identify false positives or identify an incident). The report shows historical trends and anonymised benchmarks against similar, aggregated metrics, obtained from other customers.



--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---





Friday, November 30, 2012

HPL Situational Awareness-as-a-Service, in the Cloud

In the context of the HPL Safe Cloud project, I have been working on an HP Labs R&D demonstrator, jointly with HP businesses, to illustrate:


• Next generation Business Operation Centers in Disaggregated IT scenarios, i.e. where an organisation relies on service providers (SaaS) and infrastructure providers (IaaS) in the Cloud to run their IT operation

• Information Sharing as a key requirement for the organisation to improve its (security, business, etc.) situational awareness, now that it has not anymore control over their IT operations- issues and trade-offs involving information sharing, involving the company and the other stakeholders, including SaaS and IaaS providers

• Next generation war rooms

• Our vision in the areas of Safe Cloud and controlled information sharing

We have achieved an important milestone: a full working implementation is available. Additional details and a few screenshots of the public, R&D version of the demonstrator are available online.

This demonstrator is now available and can be shown to HP customers and business partners. Below I attach, as an example, a screenshot:




We focus on a scenario involving a company that increasingly relies on SaaS and IaaS Cloud Providers to run their IT Operations. The demonstrator uses advanced visualisation and back-end processing techniques to show a futuristic, next generation Business Operation Center, supporting a company to monitor/manage their disaggregated IT.

The demonstrator provides an overview of the various company's SaaS providers along with the dependencies they have on IaaS Cloud providers and the high-level “health” status of their services.

We then use the demonstrator to illustrate the need that a company has for information sharing - to enable better situational awareness - now that the company has lost control on its IT Operations. We highlight the tension-points involved in information sharing, the trade-offs that are acceptable by the various stakeholders and the consequences of sharing data.

The demonstrator shows various view points, in terms of available information and what can be shared. For example it is possible to focus on a SaaS Provider and/or an IaaS Provider, show the locally available information and which information can actually be collected, processed and shared with the company - based on agreed policies. The demonstrator highlights some of the implications of sharing data, i.e. via live metrics, highlighting risk points and related alerts.

The demonstrator can also show the dependency on the IT infrastructure used in the Cloud and various types of metrics/information that can be exchanged with the company (right - as part of a mutual agreement). This include information on IT performance, security and incident management aspects.

A key capability of the demonstrator is to enable the audience to interactively play different roles, such as acting as the company or one of the SaaS providers. A player can interact with the system and the other players, decide which information to share (for example with other SaaS providers and/or the company) in order to accomplish common goals (e.g. dealing with an incident or an attack). We believe this creates further awareness about the importance of information sharing, the implications and tension-points in doing it, and the needs for information sharing controls.

In our HP Labs vision, HP could provide these capabilities (dashboards, controlled information sharing, analytics, etc.) as a (Security) Service to its customers, for example in the context of Managed Services and/or Next generation SOCs.



--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---



HPL Security Intelligence-as-a-Service (SILAS)

As discussed in previous posts, our HPL Security Intelligence-as-a-Service (SILAS) solution consists, at the very base, of R&D Analytics Technology that provides: statistical analysis of data; predictions based on simulations.
We now have achieved an important milestone in collaboration with HP business groups: a full working implementation is available.

Additional details and a few screenshots of the public, R&D version of SILAS are available online. Below I attach a screenshot of the SILAS main dashboard.





A typical scenario (where SILAS can be deployed and add value) consists of a multitenant Security Operation Center (SOC),

In this scenario the SOC manages incidents and IT operation issues for multiple customers. SILAS calculates and provides a wide variety of strategic metrics: customer metrics, reflecting the effectiveness of their processes (e.g. vulnerability and threat management - VTM, identity and access management - IAM, etc.), based on the data they shared with the SOC; metrics related to external threat environments (e.g. derived from information collected from HP ArchSight, HP TippingPoint, DV Labs, OSVDB, etc.); metrics providing an assessment of SOC processes, e.g. how effectively they identify incidents, close alerts, deal with false positives; what-if analysis and predictive metrics. All these metrics can be conveyed to customers (and/or other stakeholders) via reports, by highlighting trend analysis and benchmarks.

SILAS is meant to:

• provide estimation of strategic (security, risk and business) metrics to decision makers and customers, in multi-tenancy, multi-customer contexts, such as Security Operation Centers and Cloud Operation Centers

• use these metrics to enable predictive and what-if analysis, by leveraging the HP/HPL Security Analytics Solution (based on modelling and simulation techniques)

• provide customers with strategic reports - based on processed metrics and prediction - to illustrate historical trends and benchmarks

• leverage Cloud infrastructure for data processing and metric estimations

SILAS is not meant to be a reactive, real-time analytic solution. It leverages existing solutions such as HP ArchSight, HP TippingPoint/ThreatLinq, OSVDB, etc. to gather the relevant data. As unique differentiation, it provides longer-term estimates of critical metrics and uses them to make predictions. It provides decision support capabilities to key stakeholders (risk management teams, customers, etc.). As such it nicely complement current HP SW offerings.

We are currently trialling this solution in collaboration with HP business groups.


--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---





On Security Analytics – Risk Analysis for Incident Management Processes

Thanks to all the people that contacted me with interest on how HP Labs has applied HP Security Analytics techniques for Risk Analysis in the context of Incident Management Processes. I would like to remind that this HP Labs capability (along with related technologies and know-how) have now been transferred to HP Enterprise Security Services (HP ESS). Please feel free to contact HP ESS representatives if you would like to use the service.




An example of the benefits and risk assessment capabilities that can be achieved with Security Analytics (specifically in the space of Security Operation Centers and their Incident Management Processes) has been docuemented in a recent HP Labs Technical Report has been recently called “Security Analytics – Risk Analysis for an Organisation’s Incident Management Processes”:



“This document is an example of the type of report an organisation would receive at the end of a HP Security Analytics engagement. The focus is on the analysis of the security risks and performance of the organisation’s Security Incident & Events Management (SIEM) Processes and related Security Operation Centre (SOC)’s activities. HP Labs carried out the underlying R&D work in collaboration with HP Enterprise Security Services and involved analysis of processes, probabilistic modeling, simulation and “what-if” analysis for some of HP’s key customers. The outcome of this was a set of case studies from which we have been able to create this more general anonymised report illustrating the richness of the risk assessment and “what-if” analysis that has been carried out.

The lifecycle management of security is critical for organisations to protect their key assets, ensure a correct security posture and deal with emerging risks and threats. It involves various steps, usually carried out on an ongoing, regular basis, including: risk assessment; policy definition; deployment of controls within the IT infrastructure; monitoring and governance. In this context, Security Incident & Events Management play a key role. Even the best information security practices and investments in security controls cannot guarantee that intrusions – accidental and criminal activities – and/or other malicious acts will not happen. Controls can fail, be bypassed or become inadequate over time; new threats emerge. Managing such incidents requires detective and corrective controls to minimise adverse impacts, gather evidence, and learn from previous situations in order to improve over time. These incident management processes are usually run in the context of a SOC and/or as part of specialised Computer Security Incident Response Teams (CSIRTS), built on top of SOCs.

Even with SIEM in place, a potential major risk for the organisation arises due to delays introduced in assessing and handling known incidents: this may postpone the successful resolution of critical security incidents (e.g. devices exposed on the Internet, exploitation of privileged accounts, deployed malware, etc.) and allow for further exploitation. Another related risk can be introduced by sudden and/or progressive changes of the threat landscape, due to changing economic and social scenarios, new business activities or process failings within the existing IT services. This might create unexpected volumes of new events and alerts to be processed by the security team and as such, introduce additional delays. Hence, it is important for an organisation to understand the risk exposure due to their Incident Management processes, explore potential future scenarios (e.g. changes in available resources or threats landscapes or adoption of Cloud solutions) and identify suitable ways to address related issues, e.g. by introducing process changes and/or making investments in security controls.

HP Security Analytics is uniquely positioned to provide the analysis of the involved risks, explore what-if scenarios and provide decision support for decision makers. This type of Security Analytics assessments is now available as a service, provided by HP ESS.”.



--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---

On HP Secure Boardroom

You might be interested in learning more about HP Secure Boardroom:
“Data and information is your enterprise's most valuable assets. Are your current security polices fully protecting them? Gain insight into a comprehensive security strategy that is adaptive to new security threats, reduces risk, and lowers TCO. Watch now to learn the logistics of this innovative approach to enterprise security”

A Videocast is available here.



--- Posted by Marco Casassa Mont (here and here) ---

--- NOTE: use this mirror blog if you prefer posting on an external blog site ---

--- NOTE: my original HP blog can be found here ---