Denver Equifax SO

Presented by: Christopher Crowley

Presentation
Resource Thumbnail

The Denver Equifax SO presentation focuses on security operations and avoiding being the next Equifax, with a core emphasis on the Analysis of Competing Hypotheses (ACH) methodology. Key concepts include the importance of decomposing complex problems, externalizing mental machinery biases, and using structured analysis to identify the most likely explanations for security incidents.

Full Transcript / Content

--- Page 1 --- 2019-07 Security Operations to Avoid Being the Next Equifax Christopher Crowley - CCrowMontance --- Page 2 --- Copyright Christopher Crowley Security Operations 2 Introduction --- Page 3 --- Copyright Christopher Crowley Security Operations 3 Christopher Crowley - Background: Had root on most systems in employer at 15 years old (Not much #CYBER in the 80s) - Sectors: Defense, Education, Energy, Government, Financial, Software Development, Telecom - Regions: US, Europe, Middle East, Asia, Australia - Currently: Consultant, author of (SANS deprecated) MGT517. Which is now: SecOps (soc-class.com), SANS: SEC511, SEC575, SEC504, … - SOC build timeline project: https://www.montance.com/soc/timeline SANS Senior Instructor Twitter: CCrowMontance --- Page 4 --- Copyright Christopher Crowley Security Operations 4 Presentation Available to Download https://mgt517.com/soc You can take photos & tweet if that’s your thing, but I would really like you to pay attention to me, which is why I’m making this slide deck available to you Stop taking notes --- Page 5 --- Copyright Christopher Crowley Security Operations 5 Terminology & Background --- Page 6 --- Copyright Christopher Crowley Security Operations 6 How This Will Work - I’m going to give you some background so you understand where I’m coming from - This is a lot of terms, technology, mindset, etc. - It will be helpful when I get into the story that I want to tell you My Approach --- Page 7 --- Copyright Christopher Crowley Security Operations 7 Full Spectrum Capability - The US DOD Defense Science Board describes adversaries in six tiers - The top tier attackers are nations state funded, and are in the “Billion Dollar Club” in capability of spending - This type of attacker can bring the full capability of traditional intelligence capability Threat Taxonomy --- Page 8 --- Copyright Christopher Crowley Security Operations 8 Examples - MITRE / Carson Zimmerman: https://mgt517.com/csoc-mitre CSOC – Cyber Security Operations Center --- Page 9 --- Copyright Christopher Crowley Security Operations 9 Functional Areas of a SOC Crowley Defined Steering Committee Command Center Network Security Monitoring Threat Intelligence Incident Response Forensics Self-Assessment --- Page 10 --- Copyright Christopher Crowley Security Operations 10 Models of Analysis Weltanschauung - Caltagirone, Pendergrast, Betz - The Diamond Model of Intrusion Analysis - All events have the elements: adversary, infrastructure, capability, and victim - Edge connected, sequenced graphs of these nodes are Activity threads - Events are described by tuples with feature and confidence E = {f1,c1; f2,c2;…} Diamond Model --- Page 11 --- Copyright Christopher Crowley Security Operations 11 Models of Analysis Weltanschauung - Hutchins, Cloppert, Amin published Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains - Based on military targeting doctrine of - Find, Fix, Track, Target, Engage, and Assess - Intrusion Kill Chain is: Reconnaissance, Weaponization, Delivery, Exploitation, Installation, Command and Control, Actions on Objectives Phases In Diamond Model: Kill Chain --- Page 12 --- Copyright Christopher Crowley Security Operations 12 Diamond Model What Else? Interruption: Beyond Prevent/Detect Adversary Interruption Explanation Detect Enhanced methods for identifying the presence of adversary activity. Degrade Reduce the ability of the adversary to perform operations, but not able to completely stop actions. Disrupt Stop adversary action while it is occurring. Deny Prevent adversary from completing an action. Deceive Provide misinformation to adversary to cause expenditure of resources, cause mistakes, and/or facilitate detection. --- Page 13 --- Copyright Christopher Crowley Security Operations 13 Phin Phisher Defense Hack Back - A DIY Guide Phineas Phisher vs. Hacking Team – Diamond Model - Initial Entry Kill Chain Phases External Facing Appliance Notes Interruption? Reconnaissance A: Phin; I: compromised assets; C: network scans; V: HT network Scanning performed Weaponization A: Phin; I: unknown; C: exploit development; V: appliance OS Develops custom exploit Delivery uncertain Exploitation uncertain Gather more info? Installation A; I: custom firmware;C: firmware ;V: appliance Custom firmware Command and Control A; I: DNS servers; C: remote access; V: appliance DNS Control Channels Action on Objectives A; I: Network access in HT DMZ; C: firmware implant; V: appliance Initial entry grants access to environment --- Page 14 --- Copyright Christopher Crowley Security Operations 14 Cultural Dimensions Weltanschauung - Power Distance Index (PDI) - Individualism versus Collectivism (IDV) - Masculinity versus Femininity (MAS) - Uncertainty Avoidance Index (UAI) - Long Term Orientation versus Short Term Normative Orientation (LTO) - Indulgence versus Restraint (IND) Geert Hofstede – National Culture --- Page 15 --- Copyright Christopher Crowley Security Operations 15 Cultural Dimensions Weltanschauung - Means-oriented vs. Goal-oriented - Internally driven vs. Externally driven - Easygoing work discipline vs. Strict work discipline - Local vs. Professional - Open system vs. Closed system - Employee-oriented vs. Work-oriented - Degree of acceptance of leadership style - Degree of identification with your organization Geert Hofstede – Organizational Culture --- Page 16 --- Copyright Christopher Crowley Security Operations 16 Sequence of Equifax Events --- Page 17 --- Copyright Christopher Crowley Security Operations 17 Walk Through - Next we’ll talk through what is known of the sequence of events - I’ll make important points as we go - But after the full sequence we’ll review lessons learned in detail - This is all in 2017 (except the SSL certificate) Will Discuss Lessons Learned Afterward --- Page 18 --- Copyright Christopher Crowley Security Operations 18 Sequence of Events Equifax, 2017 Jan 31, 2016 Certificate allowing SSL visibility expires Feb 14 Struts vulnerability identified Mar 7 Struts Public Disclosure - CVE-2017-5638 – CVSS BASE: 10.0 --- Page 19 --- Copyright Christopher Crowley Security Operations 19 Sequence of Events Equifax, 2017 Mar 8 Equifax notified by US-CERT (Probably generic notice to many orgs) Mar 9 Internal Equifax notification: Patch Struts within 48 hours Mar 10 Post exploit discovery reveals this is initial entry date / time by intruder (48 hours) --- Page 20 --- Copyright Christopher Crowley Security Operations 20 Sequence of Events Equifax, 2017 Mar 14 Internal Emerging Threats team releases Snort rule to identify attack traffic. Countermeasures team installs it Mar 15 Updated McAfee signatures provides check for Struts vulnerability. Scans conducted, no vulnerabilities reported Mar 16 Global Vulnerability Team briefs on vulnerability and tells teams to patch, everyone sent copies of slides --- Page 21 --- Copyright Christopher Crowley Security Operations 21 Sequence of Events Equifax, 2017 May 13 Struts front end to ACIS system – ACIS houses customer information, and hasn’t been updated in decades May 13 – July 30 Attackers gain initial access via Struts, maintain access using installed webshells Infra Two application servers, two web servers Firewall protecting network segment, but web servers exposed --- Page 22 --- Copyright Christopher Crowley Security Operations 22 Infrastructure Diagram (Simplified) --- Page 23 --- Copyright Christopher Crowley Security Operations 23 Sequence of Events Equifax, 2017 May 13 - July 30 Web shells installed on both application servers from web servers May 13 – July 30 Accessible file share mounted, contained file with plain text credentials (contrary to policy) File shares not segmented within “legacy environment” --- Page 24 --- Copyright Christopher Crowley Security Operations 24 Sequence of Events Equifax, 2017 May 13 – July 30 Credentials stolen from file shares used to access database servers Segmentation of databases not performed, all Equifax DBs available from within ACIS May 13 – July 30 Access to the data base servers included queries for meta data to identify the information found within tables 9,000 total queries performed, undetected and unchallenged --- Page 25 --- Copyright Christopher Crowley Security Operations 25 Sequence of Events Equifax, 2017 May 13 – July 30 9,000 queries, 265 had PII information No secondary protection such as encryption within the database for PII data Infra Visibility for monitoring was expected to be performed by SSL intercept capable inspection of traffic into the ACIS network --- Page 26 --- Copyright Christopher Crowley Security Operations 26 Sequence of Events Equifax, 2017 (January 31, 2016) Certificate allowing SSL visibility expires --- Page 27 --- Copyright Christopher Crowley Security Operations 27 Sequence of Events Equifax, 2017 July 29 Countermeasures team installs new certificates to SSLV system (67 new certs) July 29 Monitoring occurs, due to newly returned visibility July 29 Obvious unauthorized network connection from ACIS environment to IP address in China --- Page 28 --- Copyright Christopher Crowley Security Operations 28 Sequence of Events Equifax, 2017 July 29 Full PCAP tool used to inspect traffic related to specific IP address July 29 10 MB of data transferred during this session No previous full PCAP available due to SSL cert July 29 ISP for this IP address blocked --- Page 29 --- Copyright Christopher Crowley Security Operations 29 Sequence of Events Equifax, 2017 July 30 Vulnerability Scanning of ACIS environment performed July 30 Vulnerability scan results indicate SQL injection flaws, and insecure direct object reference issues July 30 Forensic investigation of hosts indicates that PII was access by unauthorized resources --- Page 30 --- Copyright Christopher Crowley Security Operations 30 Sequence of Events Equifax, 2017 July 30 Second malicious IP address – in Germany identified as accessing ACIS environment 12:41pm ACIS environment taken offline “for emergency maintenance.” July 30 Executives and Senior Executives briefed on status of breach --- Page 31 --- Copyright Christopher Crowley Security Operations 31 Sequence of Events Equifax, 2017 ~Aug 14 Effort of 50-60 people start working on remediation effort (Project Sparta) Substantial effort put forth to develop customer facing portal Sept 4 143 Million affected users identified --- Page 32 --- Copyright Christopher Crowley Security Operations 32 Sequence of Events Equifax, 2017 Sept 7 Public notification Website, Call Centers Formal notification to all 50 states --- Page 33 --- Copyright Christopher Crowley Security Operations 33 Sequence of Events Equifax, 2017 Sept 7 - 30 Website clones generated Official Equifax twitter tweets link to wrong site Mis-information provided to customers Website intermittently unavailable, code inefficiencies due to short fuse development --- Page 34 --- Copyright Christopher Crowley Security Operations 34 Sequence of Events Equifax, 2017 Sept 15 CISO and CSO retire Sept 26 CEO Retires Oct 2 Forensic investigation finalized, number increases from 143 to 145.5 CIO for Global Platforms (ACIS) fired --- Page 35 --- Copyright Christopher Crowley Security Operations 35 Sequence of Events Equifax, 2017 Early 2018 Affected individuals calculation rises to ~148 million --- Page 36 --- Copyright Christopher Crowley Security Operations 36 Sequence of Events Equifax, 2017 Final Details of Data Breach - Final Stats - Name: 146.6M - DOB: 146.6M - SSN: 145.5M g@M3 ()\/3R  --- Page 37 --- Copyright Christopher Crowley Security Operations 37 Lessons Learned from Equifax --- Page 38 --- Copyright Christopher Crowley Security Operations 38 Lessons Learned - When there are lots of items like this, I try to enumerate them in an ordered fashion, so I’m going to go through the timeline above and point out some items. Would take days to consider and report all - I typically would do this with Kill Chain or Diamond Model to explain the intervention opportunities, next slide is a repeat example from another famous compromise: Phin Phisher attacking Hacking Team Where to Start? --- Page 39 --- Copyright Christopher Crowley Security Operations 39 Phin Phisher Defense Hack Back - A DIY Guide Phineas Phisher vs. Hacking Team – Diamond Model - Initial Entry Kill Chain Phases External Facing Appliance Notes Interruption? Reconnaissance A: Phin; I: compromised assets; C: network scans; V: HT network Scanning performed Weaponization A: Phin; I: unknown; C: exploit development; V: appliance OS Develops custom exploit Delivery uncertain Exploitation uncertain Gather more info? Installation A; I: custom firmware;C: firmware ;V: appliance Custom firmware Command and Control A; I: DNS servers; C: remote access; V: appliance DNS Control Channels Action on Objectives A; I: Network access in HT DMZ; C: firmware implant; V: appliance Initial entry grants access to environment --- Page 40 --- Copyright Christopher Crowley Security Operations 40 Lessons Learned - A good way to reinforce a lesson is to review it shortly after the lesson, to reinforce the thoughts - If you want, take this slide deck, go through the preceding Equifax sequence and enumerate suggested interruption opportunities - In the interest of time, I’m going to discuss some of them in the following slides - Feel free to send your results to me: chris @ montance.com Your Homework --- Page 41 --- Copyright Christopher Crowley Security Operations 41 Reminder: Diamond Model What Else? Interruption: Beyond Prevent/Detect Adversary Interruption Explanation Detect Enhanced methods for identifying the presence of adversary activity. Degrade Reduce the ability of the adversary to perform operations, but not able to completely stop actions. Disrupt Stop adversary action while it is occurring. Deny Prevent adversary from completing an action. Deceive Provide misinformation to adversary to cause expenditure of resources, cause mistakes, and/or facilitate detection. --- Page 42 --- Copyright Christopher Crowley Security Operations 42 (January 31, 2016) Certificate allowing SSL visibility expires Interruption Opportunities Sequence of Events Equifax, 2017 Detect: Scan for certificates Detect: SIEM feed health Disrupt: Web app firewall Deny: Fail Closed --- Page 43 --- Copyright Christopher Crowley Security Operations 43 Zimmerman: Coverage - SIEM report assessing the feed performance - Would have easily identified the SSL visibility lapse Coverage Report Example --- Page 44 --- Copyright Christopher Crowley Security Operations 44 Crowley Incident Avoidability - On a (discrete) scale of 1,2,3, was it avoidable? - 1 means a measure, already available in the environment wasn’t applied and resulted in the incident - 2 means a measure is available in the larger environment and something (economic, political) prevents implementing it within the organization - 3 means nothing is available to prevent that method of attack - Assess every issue identified in this way Simple: 1, 2, 3 1 - Patch available, but not applied 2 - Patch available, risk decision not to apply 3 - No patch available (zero day) --- Page 45 --- Copyright Christopher Crowley Security Operations 45 Crowley Incident Avoidability Equifax Score: 1 1 - Patch available, but not applied --- Page 46 --- Copyright Christopher Crowley Security Operations 46 Crowley Incident Avoidability - You don’t care about US Congress (or Equifax Board of Directors), but when you have to go explain what happened to whatever authority you answer to, do you want to be explaining why you were a 1? - They were all CYA, fired this person, we’re making improvements, etc. Equifax Score: 1 1 - Patch available, but not applied --- Page 47 --- Copyright Christopher Crowley Security Operations 47 Impact Level Definition - Few systems (or only a specific type) - Unimportant systems - Unimportant dataLow - More systems (or many common types) - Important or high value person’s, account, or system - Important data at riskModerate - Most systems (or almost all types) - Highest level accounts, users, and systems - Business critical dataHigh Does Your Management Understand This Already? --- Page 48 --- Copyright Christopher Crowley Security Operations 48 Impact Category Definition - Low – minimal function disruption - Moderate – substantial disruption - High – complete disruption Functional - Intellectual Property (L/M/H) - Integrity Manipulation (L/M/H) - Privacy violated (such as PII / PHI) Informational - Regular – predictable using resources on hand - Supplemented – predictable with augmented resources - Unrecoverable – data breach which cannot be undone Recoverable Does Your Management Understand This Already? --- Page 49 --- Copyright Christopher Crowley Security Operations 49 Scale: 1-10 3 7 10 Low Moderate High 3 Functional 9 21 30 7 Informational 21 49 70 10 Recoverable 30 70 100 Impact Quantification Example Quantification Current Incident 191 Legacy System #8 - ACIS --- Page 50 --- Copyright Christopher Crowley Security Operations 50 More Metrics? - Carson Zimmerman and I co-authored a talk that he gave at FIRST last month: - https://mgt517.com/first-metrics If You Like Metrics --- Page 51 --- Copyright Christopher Crowley Security Operations 51 The Key to This Talk - So, take the most important asset you have, and think about how that isn’t being protected effectively - Assume that protection will fail, then think about how you can prevent catastrophic failure, as well as what you’ll do when failure (small or large) happens - Let’s check out how Equifax performed when they identified the issue Your Company, Like Equifax, Has Something of Value --- Page 52 --- Copyright Christopher Crowley Security Operations 52 Communication Is Critical Don’t Create a New Domain During a Security Incident - First version of Equifax imposter page, courtesy, internet archive --- Page 53 --- Copyright Christopher Crowley Security Operations 53 Equifax Dumpster Fire - Again, don’t create a new website - If you do, be sure you tweet the correct address: What Not to Do During a Security Incident https://www.equifaxsecurity2017.com/ 53 --- Page 54 --- Copyright Christopher Crowley Security Operations 54 Resolution to Asset Discovery - System recording the responsible party for each system - Owner - Contact for Operations - Data experts - Boundaries - Scenarios of compromise (actual past, or industry specific) - Risk management - Incident handling specific details - Public relations point of contact - Legal requirements and notification Fundamental Error – No Culpability/Responsibility: Until After Breach --- Page 55 --- Copyright Christopher Crowley Security Operations 55 Does Anyone Do Asset Discovery? - Seriously, is the technology incapable of identifying systems on your network? - I think the disappointment is mapping to useful organizational contact A Technology Problem? --- Page 56 --- Copyright Christopher Crowley Security Operations 56 Matrix Depiction Network Infrastructure - Owner: Dir Network - POC: Thomas Racks - Impact: Moderate - High Server Infrastructure - Owner: CIO - POC: Thomas Racks - Impact: Low - High Workstations - Owner: Dir Network - POC: Thomas Racks - Impact: Moderate – High Financial - Owner: CFO - POC: Sarah Banks - Impact: Moderate - High JABB - Owner: BB Acct POC - POC: Vincent Lehman - Impact: Moderate - High AMD - Owner: AMD Acct POC - POC: Alexandra Harris - Impact: Moderate – High GCCS - Owner: GCCS Acct POC - POC: Victoria Houston - Impact: Moderate – High MEH - Owner: MEH Acct POC - POC: Iggy Calendula - Impact: Low - Moderate Generic Server System - Owner: CIO - POC: SOC Manager - Impact: Low – High Generic System - Owner: CIO - POC: SOC Manager - Impact: Low – High Generic Cloud Hosted System - Owner: CIO - POC: SOC Manager - Impact: Low - High Part 1 - Capabilities Mapped to System Types - Centralized knowledge base of known and unknown system types: reporting requirements, service level objectives (SLO), POCs, Strategies, reporting SLAs, customers, cyber insurance - Handling strategies, guidance for impact determination, IP address ranges, operating system types, exceptions, known vulnerabilities, previous incident types, associated exercises, response capabilities applied based on per-system and per-incident type - Internal dependencies, cloud contracts, external auditors, legal council --- Page 57 --- Copyright Christopher Crowley Security Operations 57 Matrix Depiction Internal Capability Recon Weapon- ization Delivery Exploit Install Command & Control Action on Objectives Spam Gateway + -- ++ +- +- -- - EDR - -- + + + + +- NIDS - -- + +- + + - Network Proxy + - ++ - +- ++ + IR - Containment - -- - - - + + Part 2 – Defense Capabilities List Versus Kill Chain Phase - List your capabilities against attacker cyber threat kill chain (or ATT&CK) phases, to know what you have at your disposal in unfamiliar circumstances --- Page 58 --- Copyright Christopher Crowley Security Operations 58 Matrix Depiction – Invincea Example - Here Invincea has listed the NIST Cyber Security Framework (CSF) and identified adversary TTPs (and specific targets) cross walked to capabilities and technology --- Page 59 --- Copyright Christopher Crowley Security Operations 59 Equifax Issue: Sprawling IT Product Regional Global Global Headquarters APAC Product 1 Product 2 Europe Product 3 Product 4 In fo r m at io n D ir ec ti o n --- Page 60 --- Copyright Christopher Crowley Security Operations 60 Response Capability Empowerment - Does the police department ask permission to cordon off an area, or drive cars fast to chase criminals? Yes, they absolutely do. They have protocols, requirements, and consideration for safety of innocent people. - Identify the authority to conduct response on each type of system, and what is to be done for unknown systems for containment. - Use business continuity plans for eradication and remediation: don’t create new plans unless no business continuity plan exists. --- Page 61 --- Copyright Christopher Crowley Security Operations 61 Response Capability Empowerment - Three characters of incident response - Janitorial Services - Fire Fighters - Apex Predators - https://www.mgt517.com/ir3 --- Page 62 --- Copyright Christopher Crowley Security Operations 62 Technology Selection Issues - Inadequate “glue” to connect the different tools - Poor selection of tools for the given purpose - Often a result of buying tools prior to understanding the intended purposes - Protection scenarios expressed as “use cases” - Video of Crowley’s presentation on use case development from 2017 SOC Brief is available: https://mgt517.com/use-case (registration required) --- Page 63 --- Copyright Christopher Crowley Security Operations 63 Technology Selection Issues - Whatever technologies are selected, each changes with some frequency and must be kept up to date - SIEM in particular, but all other tools require dedicated staff to keep it tuned and operating effectively. Dedicate time for this - This is sometimes transferred to a managed service provider (MSP/MSSP) --- Page 64 --- Copyright Christopher Crowley Security Operations 64 Technology Maintenance Issues - SOC relation to standard IT? - Oversight - Embedded - Coordinating - Figure out what you’re going for - Sequence of growth/maturity is often: - Embedded (security activity in name only) - Coordination / Oversight (separate to grow) - Embedded (mature organization with clear picture of what is needed and optimizes effort) Oversight Coordination Embedded --- Page 65 --- Copyright Christopher Crowley Security Operations 65 Resolutions: Security Staff Empowerment & Reality Check --- Page 66 --- Copyright Christopher Crowley Security Operations 66 Training, Exercise, and Practice - There’s a large group of people responsible that allowed that TLS certificate to expire - It’s some mix of cultural laziness, powerlessness, pride, meekness, etc. but they missed it for a long, long time - Security failed at basic IT Operations Someone, Some Group, Some Manager, Some Executive… --- Page 67 --- Copyright Christopher Crowley Security Operations 67 Training, Exercise, and Practice - Seek funding for external training - Cultivate internal training development - Practice tasks – required self-improvement time - Scripts, Internal tools, Coding, Mess remediation – 2 hours per week, required, for example - Encourage external presentation by paying for travel to conferences if staff are approved for presentations - Includes time for threat hunting, and other non-alert driven work --- Page 68 --- Copyright Christopher Crowley Security Operations 68 Training, Exercise, and Practice - You should develop training & certification plans per staff role - Each role has certs that are - Required for hire (or completion within x days of hire) - Standard training plan sequence - Optional to training plan (but company will pay for) Simple or Complex Training Plan --- Page 69 --- Copyright Christopher Crowley Security Operations 69 Monthly Staff Presentations - SOC staff rotates on presenting current technique, issue or problem - This can be an unresolved problem, seeking input and assistance from all SOC areas - Opportunity to share methodology, thought processes, and tools - Should be a formal presentation (slides, Q+A) Self-Improvement, Group Format --- Page 70 --- Copyright Christopher Crowley Security Operations 70 Quarterly Internal Training - External training is limited due to lack of time and/or funding - Resolution: Develop internal exercises - Quarterly release of new exercise - Based on current issues and events - Additionally, results in exercises useful for: interviews, onboarding, and keeping staff sharp Group Self-Challenge --- Page 71 --- Copyright Christopher Crowley Security Operations 71 Analytical Methodology - Almost no organizations that I encounter have a methodology in place for performing analysis - There should be a training on analysis as well – few places have this - SANS SEC511 has some of it, ChrisSanders.org has an analysis class online - I’ll discuss Analysis of Competing of Hypothesis (ACH) next Group Methodology for Collaborative Performance --- Page 72 --- Copyright Christopher Crowley Security Operations 72 Models of Analysis Background - Human mental machinery is fraught with substantial weakness in objectively processing information - We overlook or ignore details that don’t suit our mental models - We resist changing our mind once we’ve decided something is right, even in the face of updated evidence - Experimental results suggest that: experts in their field increase self-assessed confidence level with more data, without corresponding increases in accuracy of analysis ACH – Mental Machinery --- Page 73 --- Copyright Christopher Crowley Security Operations 73 Models of Analysis Background - We use Situational Logic, Theory Application, and Comparison - Leads to mistakes in analysis - Make assumptions, but explicitly identify them in the analysis - Common analytical failures: - Satisficing – selecting the first, not the best - Incrementalism – variation on same theme, insufficient breadth of scope - Consensus – most popular, not the most accurate - Reason by analogy – selection based on avoiding a previous mistake in analysis - Principals – applying morality to select between the “good” and “bad” ACH – Mental Machinery --- Page 74 --- Copyright Christopher Crowley Security Operations 74 Analysis of Competing Hypotheses Background - Decomposing a problem breaks it into sub-elements to allow smaller units for analysis - Externalization of a problem removes it from the mental space it puts it in a tangible space to facilitate more complex manipulation. Draw a graph, build a model, use replicas of the affected systems and test the scenarios you propose ACH – Decomposition and Externalization --- Page 75 --- Copyright Christopher Crowley Security Operations 75 Analysis of Competing Hypotheses Background Incident Type Hypothesis 1 Hypothesis 2 Hypothesis 3 … Data 1 Data 2 Data 3 Data 4 … What’s Your Template for Analysis? We’re going to develop these templates. You’re not just to fill out the template, you still need to think! --- Page 76 --- Copyright Christopher Crowley Security Operations 76 Analysis of Competing Hypotheses ACH Applied - Let’s look at the data we have, and list some potential explanations - Don’t criticize, kill, or otherwise refute any hypothesis! (We’ll do that later.) - This is hard for most people to do, since we tend to be incremental - Automation: populate hypotheses with all previously encountered explanations for issues 1. Brainstorm --- Page 77 --- Copyright Christopher Crowley Security Operations 77 Analysis of Competing Hypotheses ACH Applied - Identify the data that we have - Identify data that would be useful, and keep a running list of it to try to acquire it - Orchestrate: Based on the attempt to build out your ACH templates, determine what data would be needed for analysis, and work toward automatically collecting this information through orchestration 2. List Data Attributes --- Page 78 --- Copyright Christopher Crowley Security Operations 78 Analysis of Competing Hypotheses ACH Applied - Make a chart with the hypotheses on the top and the data elements (created in steps 1 and 2) on the side - Don’t fill it in yet - At some future time, this could potentially be automated for an analyst given ranked / weighted hypotheses sets based on data present for investigation, (we’re probably not there yet) 3. Build Matrix --- Page 79 --- Copyright Christopher Crowley Security Operations 79 Analysis of Competing Hypotheses ACH Applied - Look for consolidation opportunities - Are two hypotheses essentially the same? - Can we consolidate data elements? - Do we need to add more data elements? - Don’t Automate or Orchestrate: Get the human being to do expert work here 4. Assess and Refine Chart --- Page 80 --- Copyright Christopher Crowley Security Operations 80 Analysis of Competing Hypotheses ACH Applied - Fill in the numbers - The method we’ll use is: 10 points per row total. Assign the points for that specific data element (not overall) that supports the hypothesis in each column) - Can’t exceed 10 points per row - Sum the points at the bottom 6. Tentative Conclusions --- Page 81 --- Copyright Christopher Crowley Security Operations 81 Analysis of Competing Hypotheses Background Malware drive by download attempt Phone actually infected Targeted not from site, but from network user is on Site Accessed when ad showed 5 2 3 Bad Grammar 3 3 3 Compulsion to risky action 3.5 3 3.5 Advertised only on one site 7 1 2 Threatening? 3.5 3 3.5 Reputation weight of domain? 4 3 3 Total: 26 15 18 ACH – Example – Spam Advertising These values are the analysis, different analysts would have different values --- Page 82 --- Copyright Christopher Crowley Security Operations 82 Analysis of Competing Hypotheses ACH Applied - Look at the matrix. Identify all the rows where the point allocation is close (3,4,4, for example) - Look at the matrix. Identify the rows where there’s a big difference between points (10,0,0 or 7,2,1). These are the data elements that your analysis hinges upon. - Consider: what if that data was wrong? (Analysts dislike) - This could be automated, but the benefit of this is for the analyst to reflect on the conclusions 6. Sensitivity Analysis --- Page 83 --- Copyright Christopher Crowley Security Operations 83 Analysis of Competing Hypotheses Background Malware drive by download attempt Phone actually infected Targeted not from site, but from network user is on Site Accessed when ad showed 5 2 3 Bad Grammar 3 3 3 Compulsion to risky action 3.5 3 3.5 Advertised only on one site 7 1 2 Threatening? 3.5 3 3.5 Reputation weight of domain? 4 3 3 Total: 26 15 18 --- Page 84 --- Copyright Christopher Crowley Security Operations 84 Analysis of Competing Hypotheses ACH Applied - Write a brief explanation of your results (potential for automation and orchestration here) - Identify most likely hypothesis - Identify your assumptions - Identify the sensitivity of your analysis (what does it depend upon being true?) - Identify other hypotheses considered but dismissed 7. Report Conclusions --- Page 85 --- Copyright Christopher Crowley Security Operations 85 Analysis of Competing Hypotheses ACH Applied - Identify a few things that are likely to happen in the future, based on what you’ve hypothesized is the most likely explanation - This isn’t easy. For example, if you hypothesized an intruder has compromised a host, there will likely be subsequent reconnaissance and pivoting attempts using exploits and password guessing 8. Milestones --- Page 86 --- Copyright Christopher Crowley Security Operations 86 Analysis of Competing Hypotheses ACH Applied - Give your completed work to your partner analyst - This isn’t escalated to a higher tier, done by a peer analyst - The partner should be able to understand what you wrote and your explanation without you explaining it to him/her – repeat: you’re not allowed to explain the report to your peer. Peer reads it, criticizes it, you improve it - Don’t agree, only point out problems that are present, this is what makes analysis better: criticize to strengthen 9. Peer Review (Crowley added) --- Page 87 --- Copyright Christopher Crowley Security Operations 87 Analysis of Competing Hypotheses Background - 1) Identify hypotheses – brainstorm - 2) List attributes – for and against each hypothesis - 3) Build matrix – with hypotheses at top, evidence on side - 4) Refine matrix – remove or consolidate as appropriate - 5) Tentative conclusions – disprove (don’t validate) item - 6) Sensitivity – change critical elements to assess - 7) Report conclusions – relative likelihood of each - 8) Milestones – future events suggesting error in analysis - 9) Peer Review Review - ACH – 8-Step ACH Process --- Page 88 --- Copyright Christopher Crowley Security Operations 88 Analysis of Competing Hypotheses Background - We can create templates for the common situations we encounter to expedite the analysis portion - There’s a danger to this: you are not supposed to just fill in the template – don’t be limited by it, it’s a start - Work on templates, and revise perpetually - Start now: 3 most frequent situations requiring analysis, and build templates for those over the next three weeks Templates – Benefits and Risk --- Page 89 --- Copyright Christopher Crowley Security Operations 89 Analysis of Competing Hypothesis Incident Type Hypothesis 1 Hypothesis 2 Hypothesis 3 … Data 1 Data 2 Data 3 Data 4 … Template Richards J Heuer – Analysis of Competing Hypothesis Book Download Available Probably a good idea to develop something like these templates for use in your environment as an analytical tool. It forces externalization and objective practice. Important – templates a work expediter, not a checkbox to fill in. --- Page 90 --- Copyright Christopher Crowley Security Operations 90 Conclusion --- Page 91 --- Copyright Christopher Crowley Security Operations 91 Thank You - CCrowMontance (twitter) - https://www.mgt517.com/soc for this slide deck & other public decks, plus additional references - Redistribution authorized, but please provide citation - https://www.montance.com/soc/timeline : current project for building a SOC - MGT517? :https://soc-class.com & https://montance.blogspot.com/2019/04/security- operations-class-status.html I Appreciate Your Time and Attention

Related Q&A

  • Q: What was the root cause of the Equifax breach?
    A: Failure to patch a known vulnerability in Apache Struts.
  • Q: What specific vulnerability was exploited?
    A: CVE-2017-5638.
  • Q: What was the 'detection deficit' in the Equifax case?
    A: Approximately 76 days.