2023 SANS SOC Survey Review: Highlights and Deep Dive (Video)

Presented by: Christopher Crowley

Presentation

The 2023 SANS SOC Survey Review provides an overview of the survey's findings, highlighting key trends and insights. The survey received over 640 responses, with an average time to complete of 59 minutes, primarily from North America, with a majority being small to medium-sized organizations. The top capabilities reported were alerting and triage, incident response, and compliance and data protection.

Featured Clips

"IF YOU DO WORK IN A SOC. IF YOU DO WORK AS AN ANALYST, IF YOU HAVE ENGINEERED OR ARCHITECTED A SOC, I WOULD APPRECIATE IT IF EITHER THIS YEAR OR NEXT YEAR YOU ACTUALLY RESPOND TO THE SURVEY."
"THIS IS NOT A FIVE-MINUTE THING THAT PEOPLE ARE JUST LIKE, YEAH, YEAH, YEAH, WHERE'S MY $50 GIFT CARD AT THE END OF IT? BECAUSE THERE IS NO GIFT CARD AT THE END OF IT."
"SOME OF THIS STUFF, YOU'RE GOING TO SEE A CHART AND IT'S JUST LIKE, OKAY, THIS IS WHAT THE ANSWERS ARE AND THEN SOME OF WHAT I SAY IS ACTUALLY GOING TO BE CHRIS'S OPINION ON WHY THAT COMES OUT THAT WAY AND WHAT PEOPLE ARE ACTUALLY DOING."

Full Transcript / Content

--- Page 1 --- Thank you very much. Thanks for coming out. I want to talk a little bit about security operations and are basically a survey that I've written for the last seven years in terms of studying of what people are doing for security operations. So just a little bit about me and why you might be interested in hearing my opinions and thoughts on people running cybersecurity operations. I work as a consultant and, you know, people typically call the style of consulting that I do like an army of one. I basically have zero employees. I do all my own contracting. I do all my own engagements. I actually do my delivery. Yes, he's like his eyes are popping. Like, why the hell would you do that to yourself? It is a little bit. It is a little bit strange, but it works for me. Okay. So I've been working in security operations for a while. Basically, I wrote a class on this. I've been writing the survey. I started working in computers when I was 15 years old, and that's in the 80s. So I was doing Unix systems administration, running tape in high school, that kind of stuff. I've been teaching for the SANS Institute and I've been a certified SANS instructor for more than 10 years. I've worked in U.S. government quite a bit, defense, energy, non-security. Defense, non-energy agencies. I've worked for a number of different organizations across a number of different sectors. And so what I've tried to do over the last several years is really get my knowledge and my understanding of the way that things work, my anecdotal experience with the way that things work into a more generalized construct of how people actually do cybersecurity operations. Okay. So if you want to connect on LinkedIn or Twitter, I'm like never on either of those, but if you send me a message there, I'll answer it in a couple of weeks. I have sort of this problem with running a SOC survey because all of the surveys that we see all the time, it's just somebody's opinion to a question that they're not really presenting data to us. So with that in mind, if I say data, please refactor that in your brain. It's not data. I've not been like measuring things. This is not this is not mass. This is not I'm not going into other people's SOCs and actually looking at the detections that they've engineered in order to be able to to comprehend it. That being said, this is probably the most open source of insight into other people's cybersecurity operations that you will find. And I've worked really hard to try to collect information in a way, collect responses in a way that I can then share that with people. In order to do something useful with it, because in my experience, a lot of people will say, OK, well, we're running our SOC this way. What is that company doing? What is that organization doing? How can I do it like them? --- Page 2 --- And people are constantly like holding their cards really close to themselves and then trying to pick other people's hands to see what they're doing. And so I'm trying to facilitate that in a way that provides some sort of a third party that's somewhat trusted. So if you do work in a SOC. IF YOU DO WORK AS AN ANALYST, IF YOU HAVE ENGINEERED OR ARCHITECTED A SOC, I WOULD APPRECIATE IT IF EITHER THIS YEAR OR NEXT YEAR YOU ACTUALLY RESPOND TO THE SURVEY. ALSO, THERE'S A RELEASE EVENT THAT'S COMING UP FOR THE 24 SURVEY THAT YOU CAN GET TO. I'M TALKING ABOUT THE DATA FROM THE PREVIOUS ONE AND THE RESPONSE PERIOD IS OPEN CURRENTLY FOR THE CURRENT ONE WHICH WILL BE RELEASED IN, I DON'T KNOW, IT'S LIKE JULY OR AUGUST OR SOMETHING LIKE THAT. THIS WEBSITE, SOC-SURVEY.COM, ALSO HAS LINKS TO THE LAST SEVEN YEARS OF REPORTS AND SINCE 2020 HAVE RELEASED THE DATA, DE-IDENTIFIED, OF COURSE, WHICH IS SOMETHING THAT I THINK IS PRETTY IMPORTANT SO THAT PEOPLE CAN LOOK AT HOW I'M DOING THE ANALYSIS AND POINT OUT AND SAY, NO, NO, NO, YOU DID THAT WRONG. THAT'S COMPLETELY WRONG. AND I'LL GET INTO SOME OF THAT IN A LITTLE BIT. SO LET ME TALK ABOUT SOME OF THE HIGHLIGHTS OF THE 23 SURVEY IN ORDER TO JUST EXPLAIN WHAT I'M SEEING. SOME OF THIS STUFF, YOU'RE GOING TO SEE A CHART AND IT'S JUST LIKE, OKAY, THIS IS WHAT THE ANSWERS ARE AND THEN SOME OF WHAT I SAY IS ACTUALLY GOING TO BE CHRIS'S OPINION ON WHY THAT COMES OUT THAT WAY AND WHAT PEOPLE ARE ACTUALLY DOING. SO AN INTERESTING THING ABOUT THE SURVEY IS THAT ROUGHLY 640 PEOPLE TOOK IT AND THE AVERAGE TIME, AND THE SURVEY IS DONE IN CALL TRICKS, THE AVERAGE TIME TO TAKE THE SURVEY WAS 59 MINUTES, EXCUSE ME, THE MEAN TIME WAS 59 MINUTES. 59 MINUTES WHERE SOMEBODY SAT THERE AND TOOK A SURVEY. THEY DIDN'T GET PAID FOR IT AND THEY SPENT AN HOUR DOING IT. NOW, THIS IS AN INTERESTING THING IN AND OF ITSELF. THE PEOPLE WHO ARE ANSWERING THIS ARE ACTUALLY INVESTING A LOT OF TIME IN IT. AND JUST IN TALKING TO PEOPLE ABOUT THIS, PEOPLE HAVE TOLD ME JUST TAKING THE SURVEY CHALLENGES THEM TO THINK ABOUT HOW THEY'RE DOING THEIR OPERATIONS. AND THE OTHER THING IS, THIS SUGGESTS TO ME THAT IT'S QUALITY RESPONSES. THIS IS NOT A FIVE-MINUTE THING THAT PEOPLE ARE JUST LIKE, YEAH, YEAH, YEAH, WHERE'S MY $50 GIFT CARD AT THE END OF IT? BECAUSE THERE IS NO GIFT CARD AT THE END OF IT. SO THIS IS ANOTHER REASON WHY THIS IS ACTUALLY SOMETHING THAT'S USEFUL. I THINK THAT YOU COULD TAKE THIS, AND THEN PEOPLE HAVE TOLD ME THEY TAKE IT AND GET BUDGET BASED ON, HEY, THIS IS WHAT THEY'RE SAYING IS USEFUL IN OUR ENVIRONMENT. ANOTHER THING THAT'S QUITE USEFUL IS A TECHNOLOGY COMPARISON, AND I'LL GET TO SOME TECHNOLOGY --- Page 3 --- EXPLANATIONS LATER, WHERE YOUR PEERS HAVE GRADED IN THE SURVEY WHAT TECHNOLOGIES WORK FOR THEM, WHAT TECHNOLOGIES DON'T WORK FOR THEM, WHAT TECHNOLOGIES THEY'RE ABLE TO GET INTO PRODUCTION FULLY, PARTIALLY, OR NOT AT ALL. SO IF YOU GO LOOK AT THIS, AND YOU'RE LIKE, HEY, I WANT TO BUY THIS GREAT NEW DECEPTION CAPABILITY, GENERALLY AROUND DECEPTION, YOUR PEERS DON'T LIKE IT, AND THEY CAN'T GET IT FULLY INTO PRODUCTION. SO IF YOU STILL WANT IT, YOU MIGHT BE THINKING, OKAY, HOW DO I CONSTRAIN MY DEPLOYMENT SO THAT I'M ONLY TACTICALLY DEPLOYING IT INTO SOME SPECIFIC PLACE TO MAKE SURE THAT IT DOES EXACTLY WHAT I WANT, RATHER THAN THINKING ABOUT DOING DECEPTION GENERALLY ACROSS MY ENTIRE ENTERPRISE. SO THAT MIGHT BE A USEFUL THING THAT YOU CAN DO WITH SOME OF THIS INFORMATION. THE OTHER PART OF THIS IS THAT THIS IS PRIMARILY NORTH AMERICAN COMPANIES AND PRIMARILY SMALL COMPANIES. I MEAN, THERE WERE SOME LARGE COMPANIES, LIKE IF YOU LOOK AT THIS GRAPHIC HERE, EACH OF THESE BUILDINGS ACCOUNTS FOR BASICALLY 25 RESPONDENTS. SO IT'S SOMETHING WHERE ROUGHLY 80 ORGANIZATIONS WERE LARGER THAN 50,000 EMPLOYEES. WHO RESPONDED TO THIS. DIFFERENT ROLES. IN THE LAST COUPLE OF YEARS, WE'VE ACTUALLY EXPERIMENTED WITH DOING THE SURVEY IN DIFFERENT LANGUAGES, AND JUST AS AN EXAMPLE, WHEN WE DID IT IN SPANISH AND PORTUGUESE, THE RESPONDENTS ACTUALLY SHIFTED TO SOUTH AMERICA INSTEAD OF EUROPE AS THE SECOND GROUP FOR HEADQUARTERS. I BASICALLY JUST HAVEN'T BEEN ABLE TO KEEP THE FUNDING GOING IN ORDER TO GET THAT INTO DIFFERENT LANGUAGES. I'VE TRIED TO COMPEL THEM TO DO ARABIC AS JUST AN EXAMPLE. I DON'T KNOW IF YOU ALL ARE AWARE OF THIS, BUT THE MIDDLE EAST IS GOING THROUGH A TREMENDOUS GROWTH SPORT, AND CYBER IS PART OF THAT. SO JUST SOMETHING TO THINK ABOUT. IF YOU'RE IN SORT OF THIS AREA, IT'S NOT JUST NORTH AMERICA. THERE ARE A LOT OF PEOPLE IN DIFFERENT PLACES TRYING TO DO CYBERSECURITY OPERATIONS, AND IN MY EXPERIENCE, THEY TEND TO DO IT SOMEWHAT DIFFERENTLY DEPENDING UPON THE WAY THAT THEIR ORGANIZATIONS ARE STRUCTURED. IN TERMS OF RUNNING A SOC, I ASKED THE QUESTION, WHAT DO YOU DO? WHAT ARE YOUR CAPABILITIES? HOW DO YOU CONSIDER WHETHER IT IS A SOC OR NOT? AND SO THE TOP THINGS WERE ALERTING AND TRIAGE. THAT SEEMS SELF-EVIDENT, BUT IN FACT, THAT'S WHAT THE RESPONSES SORT OF CONFIRMED. THE INCIDENT RESPONSE CAPABILITY WAS PART OF THE SOC CAPABILITY. SOME ORGANIZATIONS WILL SPLIT THAT OUT AND SAY, NO, NO, NO, OUR INCIDENT RESPONDERS ARE OVER THERE. THIS IS OUR MONITORING CAPABILITY. COMPLIANCE AND DATA PROTECTION SORT OF ROUND OUT THE LIST OF THE DIFFERENT THINGS THAT ARE THERE. I'LL TALK ABOUT OUTSOURCING IN A MOMENT, BUT ACTUALLY THIS PARTICULAR REPRESENTATION IS INTENDED TO DEPICT IN ADDITION TO WHAT DO YOU DO, HOW DO YOU DO IT? WHAT SORT OF THINGS WERE OUTSOURCED AS OPPOSED TO DONE INTERNALLY, AND WHAT THINGS ARE YOU ACTUALLY DOING BOTH INTERNALLY AND OUTSOURCED? I'LL COME BACK TO SOME OF THOSE ASPECTS OF THE OUTSOURCING. --- Page 4 --- FOR THE PEOPLE WHO RESPONDED TO THE SURVEY, THEY SAID THAT MOST OF THEIR INCIDENTS, THEY'RE ACTUALLY ABLE TO DETECT PRIOR TO HAVING A THIRD-PARTY NOTIFICATION. AND THIS IS AN INTERESTING ONE BECAUSE WHEN YOU LOOK AT VERIZON DATA BREACH INVESTIGATION REPORT, A LOT OF WHAT THEY'RE SAYING IS THAT THE BREACHES TEND TO BE KNOWN WHEN YOU HAVE THIRD-PARTY DETECTION. AND THEN YOU'RE BEING TOLD ABOUT THAT. SO THE INFORMATION REPRESENTED IN THE RESPONDENTS TO THIS SURVEY, MOST OF WHAT THEY'RE SEEING, THEY'RE DETECTING INTERNALLY. I CAN'T NECESSARILY SAY THAT THAT IS SOMETHING THAT'S DUE TO THEIR OWN MATURITY OR THAT THE PEOPLE WHO ARE SELF-SELECTING INTO THE RESPONSES ARE ACTUALLY DOING THIS, BUT THIS IS SOMETHING THAT IS DIFFERENT THAN WHAT I'VE SEEN IN A BUNCH OF THE OTHER SORT OF INDUSTRY REPORTS OUT THERE. METRICS WERE, IT'S INTERESTING TO ME EVERY YEAR WHEN I SEE THIS, 84% SAY THAT THEY COLLECT AND EXPOSE METRICS, AND EVERY TIME I SEE THAT ANSWER, I THINK, WHAT ARE THESE OTHER 16% OF THE PEOPLE DOING? LIKE, LITERALLY, HOW DOES YOUR ORGANIZATION CONTINUE TO GIVE YOU MONEY TO RUN A SOC IF YOU'RE NOT REPORTING ANYTHING? I MEAN, THAT SEEMS CRAZY TO ME. 11% OF THEM SAID NO, WE DON'T PROVIDE METRICS. THE OTHER ROUGHLY 10%. 5% SAY THAT THEY DON'T KNOW. SO THE RESPONDENTS SAID THAT THEY WEREN'T SURE IF THEY WERE REPORTING METRICS OR NOT. OF THE METRICS THAT ARE REPORTED, AND THERE'S LOTS OF QUESTIONS AROUND THESE DIFFERENT ONES, THE TOP THREE, TOP ONE, QUANTITY OF INCIDENTS, WHICH I THINK IS A MEANINGLESS METRIC, TO BE PERFECTLY HONEST WITH YOU. HOW MANY DID YOU HAVE? WE HAD 36 LAST MONTH. IS THAT THE RIGHT NUMBER? IS THAT A GOOD NUMBER? IS THAT GOING UP? IS THAT GOING DOWN? THE METRIC ITSELF IS SOMETHING THAT IS NOT MEANINGFUL, AND YOU'RE MEASURING SOMETHING THAT IS OUT OF YOUR CONTROL IN TERMS OF YOUR DETECTIVE CAPABILITY. SO THIS IS ONE, THOUGH, THAT A LOT OF PEOPLE REPORT THAT THEY DO. TIME TO DETECT, TIME FROM DETECTION TO ERADICATION IS ANOTHER ONE THAT IS FREQUENTLY REPORTED. I THINK THAT THIS IS ACTUALLY AT LEAST SOMETHING THAT YOU CAN CONTROL FOR THE RATIO OF INCIDENTS FROM KNOWN AND UNKNOWN VULNERABILITIES. ALSO AN INTERESTING THING, THIS STARTS TO GET INTO POSTURE MANAGEMENT, CAPABILITY, VULNERABILITY MANAGEMENT, YOUR ABILITY TO ACTUALLY DETECT THINGS THAT ARE IN YOUR ENVIRONMENT, AND THEN TRACE THAT THROUGH A ROOT CAUSE TO BE ABLE TO IDENTIFY, YEAH, THIS WAS A PROBLEM THAT WE KNEW ABOUT, AND IT HURT US. SO THIS IS SOMETHING THAT IS QUITE USEFUL FOR SOCS TO DO, AND A LOT OF ORGANIZATIONS DON'T DO IT WELL, AND THE TOP METRIC BEING THE ONE THAT'S JUST LIKE A SIMPLE COUNT IS AN INDICATION OF THAT, IN MY OPINION. ANOTHER THING THAT I ASKED ABOUT IS THE IDEA OF ORCHESTRATION AUTOMATION. THIS HAS BECOME A BIT OF A BUZZWORD OVER THE LAST FIVE YEARS. ORCHESTRATION AUTOMATION REALLY IS SCRIPTING THE TOOLS IN ORDER TO DO THE --- Page 5 --- THINGS THAT WORK WITHIN A WORKFLOW THAT YOU HAVE DECIDED IS THE RIGHT WAY THAT YOUR ORGANIZATION WILL RUN YOUR SECURITY OPERATIONS. SO IN ADDITION TO THE QUESTION OF ARE YOU DOING THIS, THERE WAS A QUESTION ABOUT HOW DO YOU APPROACH THE TUNING NEEDS OF THE SOAR, WHICH I THINK IS A LITTLE BIT MORE INSIGHTFUL RATHER THAN ARE YOU USING IT, DO YOU HAVE THIS TECHNOLOGY, DO YOU LIKE THE TECHNOLOGY, IT'S HOW DO YOU TUNE IT. AND WHAT WAS INTERESTING TO ME IS THE TOP RESPONSE IS THAT, YES, WE DO ACTUALLY FREQUENTLY CHANGE OUR WORKFLOWS, AND I'LL ZOOM IN ON THIS SO YOU CAN READ IT IF YOU WANT TO SEE IT. SO WE FREQUENTLY CHANGE THE SOAR WORKFLOWS. WE HAVE DEDICATED STAFF IS ACTUALLY A LOWER ANSWER THAN WE FREQUENTLY CHANGE THE WORKFLOWS AND THE ANALYSTS OR USERS OF THE SOAR ARE AUTHORIZED AND EMPOWERED TO MAKE THE CHANGE DIRECTLY. NOW, OPTIMALLY, I THINK THAT THERE WOULD ACTUALLY BE SOME SORT OF A WORKFLOW VERIFICATION PRIOR TO AN ANALYST WHO'S IN THE MIDST OF DOING SOMETHING BEING AUTHORIZED TO CHANGE THE WORKFLOW. THAT BEING SAID, THE FACT THAT THEY SAY THAT THEY FREQUENTLY CHANGE IT AND ADJUST IT ANYWAY I THINK IS ACTUALLY A REALLY POSITIVE THING. SOAR IS EFFECTIVELY JUST APIs TO BE ABLE TO PULL DATA FROM DIFFERENT PLACES AND SCRIPTING TO DO THE THINGS THAT YOU NEED TO DO. SO IF YOUR ANALYSTS ARE EMPOWERED TO DO THE ADJUSTMENT ON THE FLY, YOU'RE GOING TO HAVE LOW MATURITY, BUT YOU ARE GOING TO BE FLEXIBLE AND ADAPTABLE. SO THIS IS SOMETHING THAT I HAVE SEEN ORGANIZATIONS DO MODERATELY WELL, AND I'M HAPPY THAT PEOPLE ARE EMBRACING THIS. THE OTHER THING I WILL SAY ABOUT SOAR, AGAIN, JUST MY OPINION ON THE DEPLOYMENT OF THIS, IS WHILE I ACTUALLY REALLY PREFER THE IDEA OF DO IT YOURSELF, WRITE THE SCRIPTS, DO ALL THESE THINGS, I THINK FROM AN ORGANIZATIONAL PERSPECTIVE, THAT IS A PATH OF PAIN IN ORDER TO DECIDE, NO, NO, NO, WE'RE JUST GOING TO DO ALL OF THAT IN PYTHON OURSELVES. AND I THINK THAT FROM AN INSTITUTIONAL ENTERPRISE PERSPECTIVE, DECIDING ON THE BEST SOAR THAT MATCHES YOUR REQUIREMENTS AND THEN DOING SMALL AMOUNTS OF CUSTOMIZATION IS ACTUALLY THE MORE EFFICIENT PATH. AND FROM SOMEBODY WHO, LIKE, IS A CONTROL FREAK, I WANT TO WRITE THE CODE MYSELF, I WANT TO VERIFY THAT IT'S BEEN SOMETHING THAT WE VETTED AND DID, I STILL THINK FROM AN INSTITUTIONAL PERSPECTIVE, THE BETTER WAY IS TO PUT IT INTO A TOOL THAT ALLOWS YOU TO DO THAT CUSTOMIZATION AND FLEXIBILITY WHEN YOU REALLY NEED IT, BUT THAT YOU DON'T HAVE TO ALWAYS WORRY ABOUT HAVING A SCRIPTING OR PROGRAMMER ON STAFF WHO'S COMPETENT TO BE ABLE TO MAKE THOSE CHANGES. ANOTHER THING THAT I ASKED ABOUT WAS THE IDEA OF HOW DO YOU DEAL WITH DATA. IF YOU'RE GOING TO DO DETECTIONS, YOU NEED TO BE ABLE TO PULL DATA INTO SOME SORT OF A CORRELATION ENGINE, TYPICALLY A SIM. --- Page 6 --- AND SO WHAT I'VE FOUND IS THAT A LOT OF PEOPLE JUST PUSH EVERYTHING INTO THE SIM AND DON'T REALLY APPLY MUCH SELECTION. AND THIS PLAYS OUT IN THE SURVEY RESPONSES, EVERYTHING GOES INTO THE SIM WAS THE TOP, AND THEN THERE'S SORT OF A RISK-BASED CRITERIA IS THE NEXT ONE. AND, I MEAN, THINK ABOUT IT, WHEN YOU ARE DECIDING DO YOU NEED THAT DATA IN YOUR SIM, THE EASY ANSWER IS YES. I'M JUST GOING TO PUSH IT IN THERE IN CASE THAT WE NEED IT. AND THIS IS, YOU KNOW, IF YOU'RE SORT OF A MAXIMALIST, YOU KNOW, PACKER OR CAMPER OR WHATEVER YOU WANT TO SAY IN TERMS OF HOW YOU PERSONALLY APPROACH THINGS, ARE YOU ALWAYS THINKING ABOUT BEING COMPLETELY MINIMAL IN TERMS OF WHAT INFORMATION YOU HAVE ON HAND? AND IF YOU'RE NOT SURE, THE EASIER SELECTION IS TO JUST THROW IT ALL IN THERE. NOW, THIS IS SOMETHING THAT SOME PEOPLE ARE SAYING. AND, ACTUALLY, ALMOST THE SAME NUMBER OF PEOPLE ARE SAYING, YEAH, WE'D ACTUALLY DO SOME SELECTION CRITERIA IN TERMS OF WHICH DATA WE'RE GOING TO COLLECT. NOW, THIS WOULD BE BETTER FROM AN EFFICIENCY PERSPECTIVE IF YOU KNOW WHAT DETECTIONS YOU'RE GOING TO ENGINEER. AND IF YOU ACTUALLY ARE DOING THAT WELL, I THINK THAT THAT'S A GOOD APPROACH TO IT. I WOULD ALSO SUGGEST THAT WITH THE ADVENT OF THE CLOUD, WITH SENTINEL AND CHRONICLE, THAT I THINK WHAT'S GOING TO HAPPEN IS A LOT MORE PEOPLE ARE GOING TO JUST START THROWING ALL THE DATA IN THERE. BECAUSE IT'S EASY TO DO. AND THEN WHEN THEY ARE BASICALLY SPENDING A LOT OF MONEY IN ORDER TO STORE DATA THAT THEY NEVER USE, EVENTUALLY IT WILL DRIVE EFFICIENCY TO BE ABLE TO SAY, NO, THIS IS THE DETECTION THAT I NEED. THIS IS THE DATA SOURCE THAT I ACTUALLY NEED FOR IT. AND LET ME GO AHEAD AND DECIDE WHICH DATA WILL BE PUSHED OUT. FOR NOW, I DON'T THINK PEOPLE HAVE FIGURED THIS OUT. AND THIS PROBABLY RESONATES WITH YOU. IT'S EASIER TO JUST PUSH IT INTO THE SIEM, PAY A LITTLE BIT OF EXTRA MONEY, NOT HAVE TO WORRY ABOUT SELECTING ON THE FRONT SIDE SO THAT YOU CAN DO QUERIES AFTERWARDS. FROM A BUDGETING PERSPECTIVE, THERE ARE A NUMBER OF THINGS THAT ARE OUT THERE IN TERMS OF HOW PEOPLE APPROACH BUDGETING. MOST PEOPLE HAVE A DIFFICULT TIME WHEN THEY'RE WORKING IN THE SOC TO UNDERSTAND HOW THAT WORKS WITHIN THEIR ORGANIZATION. AND SO THE LARGEST ANSWER WAS BASICALLY THAT THE MANAGEMENT TAKES A BUNCH OF RECOMMENDATIONS FROM THE LEADS BUT ULTIMATELY DECIDES HOW THEY'RE GOING TO ALLOCATE FUNDING. THIS IS THE REALITY THAT ALL OF US FACE. I HAVEN'T FOUND ANYBODY THAT SAYS, YEAH, WE HAVE A LIMITLESS BUDGET. THE ONLY PEOPLE THAT COME CLOSE TO THE LIMITLESS BUDGET ARE THE ONES THAT ARE BASICALLY IMMEDIATELY AFTER A LARGE-SCALE BREACH. FOR THE FIRST TWO YEARS, AND I'M NOT MAKING THIS UP, THIS IS THE CONVERSATIONS THAT I'VE HAD WITH PEOPLE CONSISTENTLY, FOR ABOUT THE --- Page 7 --- FIRST TWO YEARS AFTER A LARGE-SCALE BREACH, PRETTY MUCH YOU CAN HAVE ALL THE MONEY THAT YOU WANT. AND THEN AFTER THAT, THINGS START TO GET RAINED IN. AND THE REASON FOR THAT THAT I'VE OBSERVED IS THAT YOUR MANAGEMENT DOES NOT UNDERSTAND HOW THE ORGANIZATION EFFECTIVELY DEPLOYS ITS FUNDING. THEY GO AND ASK FOR STUFF. AND THEN THE FUNDING COMES OR IT DOESN'T COME. MAKING A COMPELLING ARGUMENT ON A CONSISTENT BASIS IN ORDER TO SHOW THIS IS THE TECHNOLOGY THAT WE NEED TO ACQUIRE. THIS IS THE STAFFING THAT WE NEED TO ACQUIRE. THIS IS THE WAY THAT THIS IS GOING TO BE EFFECTIVE AT DEFENDING AND PROVIDING LOSS PREVENTION. OVER TIME IS HOW YOU ACTUALLY SUCCESSFULLY WIN BUDGET. BUT THE SOC IS PROVIDING AN OFFER OF LOSS PREVENTION. AND I THINK THAT THESE WORDS ARE THINGS THAT YOU SHOULD THINK ABOUT HOW TO USE THEM IN YOUR ENVIRONMENT. YOU'RE NOT EVEN GUARANTEEING LOSS PREVENTION. YOU'RE PROVIDING AN OFFER OF LOSS PREVENTION. AND YOUR ORGANIZATION IS MAKING A DECISION, IS IT WORTH IT FOR ME TO SPEND ALL THIS MONEY ON SOMETHING THAT MIGHT SOLVE SOME PROBLEMS? OR IS IT BETTER FOR US TO JUST KEEP GOING AND DUMPING MONEY INTO BUSINESS INVESTMENT AND LET ALL OF THOSE OTHER THINGS JUST KIND OF FALL OUT HOW THEY WILL? AND FAIL A LITTLE BIT BUT FAIL AS GRACEFULLY AS POSSIBLE. AND IT'S NOT UNTIL THERE'S SOME BIG FAILURES THAT NOT YOUR CYBER MANAGEMENT BUT YOUR EXECUTIVE MANAGEMENT SAY YEAH, YEAH, YEAH, WE NEED TO DEAL WITH THAT. THE BEST THING THAT COULD HAPPEN TO YOU IN TERMS OF BUDGETING IS TO HAVE AN ORGANIZATION MUCH LIKE THE ORGANIZATION THAT YOU WORK IN HAVE A BIG BREACH SO YOU DON'T HAVE TO DEAL WITH THE REAL PAIN OF IT AND THEN TAKE THAT HEADLINE TO YOUR MANAGEMENT AND SAY THIS IS WHAT'S GOING ON IN THIS INDUSTRY. THIS IS HOW THEY'RE TARGETING THESE PEOPLE. THAT'S BASICALLY US. AND WILL BE THE NEXT HEADLINE IF YOU DO NOT ADDRESS SOME OF THESE FUNDAMENTAL PROBLEMS. AND I THINK THAT THIS IS THE KEY TO WINNING BUDGET. OKAY? AND I MEAN, I HESITATE TO, YOU KNOW, TO ASK ORGANIZATIONS TO SPEND MONEY WHEN THEY DON'T NEED TO. AND I THINK THAT THAT'S SOMETHING THAT WE SHOULD REALLY THINK ABOUT. WE COULD FIX EVERYTHING IF WE HAD LIMITLESS MONEY. WE SIMPLY DON'T HAVE LIMITLESS MONEY AND SO IT'S THE RIGHT SORT OF PRIORITIZATION. FOR OUR BUDGET. AND THIS IS IN LIGHT OF WHAT THE ORGANIZATION NEEDS TO DO IN ORDER TO SURVIVE. BECAUSE WE DON'T HAVE ENOUGH STAFF, BECAUSE WE DON'T HAVE ENOUGH INTERNAL CAPACITY, KNOWLEDGE, CAPABILITY, WE TEND TO LOOK FOR THIRD PARTIES TO HELP US IN ORDER TO OUTSOURCE SOME OF THE THINGS THAT WE NEED TO DO. GOING BACK TO THAT LIST OF CAPABILITY THAT I SHOWED EARLIER, THIS CHART IS A LITTLE BIT CONFUSING. SO LET ME EXPLAIN IT. THE WAY THAT THIS CHART IS ACTUALLY DEPICTED IS NOT A COUNT, BUT A RANK IN ORDER. OKAY? --- Page 8 --- SO THE RANK THAT SHOWS RED IS ACTUALLY THE ORDER OF THE SORT FROM TOP TO BOTTOM. AND THIS IS THE HIGHEST TO LEAST OUTSOURCED ACTIVITIES AND CAPABILITIES WITHIN THE ORGANIZATION. THE BLUE RANK IS WHETHER IT'S DONE AT ALL. SO LET ME JUST SORT. OF MOVE THIS IN HERE. SO IF YOU LOOK AT THE ORDER OF THE RED, BASICALLY THIS IS THE MOST OUTSOURCED TO THE LEAST OUTSOURCED. AND THE BLUE IS THE MOST LIKELY TO BE DONE TO THE LEAST LIKELY TO BE DONE. AND AT THE TOP, WHERE THE RED RANKS ARE THE THINGS THAT ARE MOST LIKELY TO BE OUTSOURCED, THERE'S A LOT OF LEAST LIKELY TO BE DONE TYPE ITEMS ON THAT LIST. PEN TESTING. RED TEAMING. PURPLE TEAMING. THE TOP THREE ACTIVITIES. BASICALLY WE'RE VERY LIKELY TO OUTSOURCE THEM IN SOCKS AND WE'RE VERY LIKELY NOT TO DO IT AT ALL. SO THIS BECOMES A CIRCUMSTANCE OF IS THE SOCK ACTUALLY VERIFYING THAT THOSE DETECTIONS AND DEFENSES THAT THEY'RE DEPLOYING WORK AT ALL? AND IF YOU'RE NOT TESTING IT ON AN ONGOING BASIS, IF YOU'RE NOT TESTING IT WHILE YOU'RE DOING THE DETECTION ENGINEERING, HOW DO YOU KNOW THAT IT WORKS? AND I'M GOING TO TELL YOU, IN MY OPINION, MOST SOCKS DON'T KNOW THAT IT WORKS. THEY DON'T. AND IT'S ONLY WHEN EITHER SOMEBODY INTERNAL DECIDES, HEY, WE SHOULD REALLY DO THIS PURPLE TEAMING THING AND MAKE THIS EFFECTIVE, THAT THAT TRACTION BECOMES SOMETHING WHERE THERE'S A DRIVE FOR CHANGE. THE OTHER THING THAT'S VERY COMMONLY OUTSOURCED, FORENSICS AND ANY SORT OF VARIATION ON FORENSICS AND THREAT INTELLIGENCE. AND THESE ARE THINGS THAT I THINK ORGANIZATIONS CURRENTLY ARE TRYING TO UNDERSTAND HOW TO INCORPORATE THAT INTO THEIR PIPELINE WITHIN THEIR SECURITY OPERATIONS, AND THEY'RE STRUGGLING. THEY'RE STRUGGLING TO BE ABLE TO UNDERSTAND HOW TO TACTICALLY DEPLOY THAT. SO THIS IS, YOU KNOW, WHAT THE WHAT THE CHART LOOKS LIKE, BUT IT'S SOMETHING THAT'S A STRUGGLE FOR EVERYBODY IN TERMS OF HOW TO DECIDE WHAT RESOURCES TO OUTSOURCE, BUT WHAT THEY'RE DOING IS USUALLY OUTSOURCING PENTESTING, FORENSICS, AND THREAT INTELL. IN TERMS OF THE SIZE OF THE SOC, THERE TENDS TO BE ABOUT 10 TO 25 PEOPLE. THIS IS BLUE, INDICATING THE MOST COMMON SIZE OF ANALYSTS, WHEREAS RED IS MORE OF THE ADMINISTRATIVE STAFF, AND THAT, THE ANSWER WAS 2 TO 10 PEOPLE. SO YOU HAVE A FEW PEOPLE, YOU KNOW, 3, 5 PEOPLE KIND OF KEEPING THE LIGHTS ON, AND THEN MAYBE, YOU KNOW, 10, 15 ANALYSTS WORKING IN ORDER TO BE ABLE TO DO THAT. I HAVE DONE A BUNCH OF DIFFERENT PIVOTS ON THIS. I'M NOT GOING TO GO INTO IT NOW. BUT WHAT I HAVE SEEN IS THAT EVEN REALLY LARGE ORGANIZATIONS TEND TO HOVER AROUND THAT 15 TO 20 STAFF. AND AGAIN, IN MY EXPERIENCE IN LOOKING AT THIS AND TALKING TO ORGANIZATIONS, LARGER ORGANIZATIONS, WHAT THEY DO IS THEY HAVE 15 STAFF MEMBERS INTERNALLY, AND THEN THOSE STAFF MEMBERS ARE ESSENTIALLY MANAGING CONTRACTORS OR INTERFACING WITH THIRD PARTIES THAT THEY TRANSFER --- Page 9 --- A LOT OF THE OPERATIONAL WORK, WHETHER YOU WANT TO CALL IT LEVEL ONE, YOU KNOW, INITIAL TRIAGE, XDR, I'M SORRY, MDR FOR YOUR END POINT DETECTION RESPONSE OR EXTENDED DETECTION RESPONSE. THIS IS SOMETHING THAT LARGER ORGANIZATIONS TEND TO LEVERAGE OUTSOURCING PRETTY SUBSTANTIALLY. FROM A STAFFING, WHAT THE STAFF LOOKS LIKE IN THE SOC, I ASKED A BIT ABOUT THE HIRING, LIKE WHAT DO MANAGERS CARE ABOUT HIRING FOR? AND SO THE ITEMS THAT WERE TYPICALLY IDENTIFIED INFORMATION SECURITY AS A GENERAL CONCEPT, THE IDEA OF THREAT ANALYSIS, DATA ANALYSIS, AND INTELLIGENCE ANALYSIS WERE THE TOP SKILLS THAT HIRING MANAGERS WERE LOOKING FOR. ASKED ABOUT HOW LONG DO EMPLOYEES STICK AROUND? THE ONE TO THREE YEARS IN THE SOC WAS THE MOST COMMON ANSWER. MOST OF YOU THIS PROBABLY RESONATES, ESPECIALLY AT THE LEVEL ONE TYPE SOC ROLE. PEOPLE WILL MOVE FOR A LITTLE BIT OF MONEY. I'M ACTUALLY GOING TO ADDRESS THAT ON THE NEXT SLIDE IN TERMS OF WHAT PEOPLE MOVE FOR AND HOW TO RETAIN THEM. BUT WHAT'S INTERESTING ABOUT THIS IS WE THEN WENT AND LOOKED ABOUT, WELL, OKAY, IN IT IS THAT NORMAL? IS ONE TO THREE YEARS NORMAL? AND IN FACT, THAT'S ABOUT THE AVERAGE TURNOVER ACROSS MOST OF IT, ONE TO THREE YEARS. SO IT'S NOT JUST CYBERSECURITY THAT HAS THIS NOTION OF CHURN, IT'S IT GENERALLY. AND SO IN ORDER TO AVOID THE NOTION OF CHURN, WHAT CAN YOU DO TO KEEP PEOPLE AROUND? THIS IS INTERESTING. CAREER PROGRESSION WAS SUBSTANTIALLY HIGHER THAN MEANINGFUL WORK OR MONEY. SO IT'S MORE COMPELLING FOR A PERSON TO STAY IN A ROLE IF THEY FEEL LIKE THEY HAVE SOMETHING THAT THEY'RE MOVING TOWARD FROM A GROWTH PERSPECTIVE. MONEY, OF COURSE, IS GOOD. I MEAN, EVERYBODY WANTS THAT. BUT THIS WAS SOMETHING BY A SUBSTANTIAL MARGIN WAS AN INDICATION THAT THAT'S HOW MANAGERS WERE ACTUALLY RETAINING STAFF. SO SOMETHING TO THINK ABOUT JUST IN TERMS OF HOW THAT MIGHT WORK. REMOTE WORK SINCE 2020 HAS BEEN SOMETHING THAT IS QUITE COMMON. AND THE QUESTION USUALLY BECOMES CAN WE SECURELY TRANSPORT THE DATA TO THE STAFF IN ORDER TO BE ABLE TO DO THAT? SOMETIMES IT'S WHETHER THE INDIVIDUAL IS ACTUALLY IN A ROLE THAT WOULD ALLOW THEM TO DO THAT. SO THIS WAS A BREAKDOWN OF WHAT MAKES IT SO THAT YOU CAN DO REMOTE WORK IF YOU'RE DOING IT. IN TERMS OF BACK TO STAFFING FROM A HIRING PERSPECTIVE, WHAT TECHNOLOGIES WERE MOST IMPORTANT FOR HIRING MANAGERS WERE EDR AND SIM. THAT'S NOT REALLY A SURPRISE, BUT IT IS SOMETHING THAT'S CONSISTENT. I HAVE THIS LONG TECHNOLOGY LIST. I'M GOING TO BASICALLY TALK ABOUT SOME OF THESE HERE. I KNOW YOU CAN'T READ IT. BUT THIS LONG TECHNOLOGY LIST WAS A RANKING OF THE GPA OF THE TECHNOLOGY. SO I ASKED THE QUESTION, DO YOU GIVE IT AN A, DO YOU GIVE IT A B, DO YOU GIVE IT A C AND SO ON? NO TECHNOLOGY GOT A'S IN TERMS OF WHAT PEOPLE GRADED IT AS. THE HIGHEST RESPONSE WAS THE EDR, EDR, XDR. I GROUPED THEM TOGETHER IN THE --- Page 10 --- QUESTIONING. THE NEXT HIGHEST WAS EMAIL FILTERS. THIS IS JUST A QUESTION OF SATISFACTION. DO YOU LIKE THIS TECHNOLOGY? DOES IT DO WHAT YOU WANT IT TO DO? SO THAT WAS AT THE TOP OF THE LIST. DOWN AT THE BOTTOM OF THE LIST, NO SURPRISE, ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING. ABSOLUTE BOTTOM OF THE LIST. NOW, THIS IS AN INTERESTING THING. AGAIN, I'M JUST MY OPINION IN TERMS OF PROJECTION. NEXT YEAR, AIML. IT WILL BE OFF OF THE BOTTOM. AND I THINK THAT OVER THE NEXT COUPLE OF YEARS, THIS WILL ACTUALLY BASICALLY CYCLE UP HIGHER INTO THE RANKING. IT WAS THE FIRST TIME THAT I HAD IT IN THE SURVEY LAST YEAR. IT WAS 23. IT WAS THE FIRST SORT OF DEBUT FOR IT. DECEPTION, FREQUENCY ANALYSIS, PACKET ANALYSIS, FULL PCAP HAVE CONSISTENTLY BEEN NEAR THE BOTTOM. THIS IS SOMETHING THAT PEOPLE DON'T LIKE, JUST FROM A PERSPECTIVE OF THAT. NOW, AGAIN, I KNOW THAT YOU CAN'T READ THIS CHART. I'M PUTTING IT UP THERE TO TALK ABOUT WHAT IT IS. THE NEXT THING THAT I DID IN THE CHART WAS TO BE ABLE TO MAP THE IDEA OF HOW FAR DO YOU GET THE DEPLOYMENT. AND SO WE ASKED THE QUESTION ON THE SAME TECHNOLOGY SET, DO YOU HAVE A FULL PRODUCTION DEPLOYMENT, A PARTIAL PRODUCTION DEPLOYMENT, ARE YOU IN SOME SORT OF A STAGING BUT HAVEN'T FULLY DEPLOYED THE DEPLOYMENT? HAVE YOU FINISHED THE DEPLOYMENT YET? AND THIS COMPLICATED CHART, WHICH, AGAIN, YOU CAN DOWNLOAD THE SURVEY AND LOOK AT THE DETAILS OF THIS, BUT LET ME PULL OUT THE IMPORTANT COMPONENTS OF IT. THIS PARTICULAR CHART, THE TENDENCY WAS THAT THE HIGHER RANKED GPA TECHNOLOGIES ARRIVED AT FULL PRODUCTION DEPLOYMENT. OKAY. SO THIS, I CANNOT DRAW A CAUSAL RELATIONSHIP BETWEEN IT, BUT IT IS TRENDING IN THE SAME DIRECTION. THE HIGHER THE SATISFACTION, THE MORE LIKELY THAT YOU HAVE A FULL PRODUCTION DEPLOYMENT. AND SO THIS IS WHAT YOUR PEERS ARE SAYING IN TERMS OF THE TECHNOLOGY AND WHAT THEY'RE DOING. I'M GOING TO SHIFT A LITTLE BIT IN TERMS OF WHAT I'M TALKING ABOUT. I'M STILL GOING TO TALK ABOUT THE SURVEY, BUT I'M GOING TO TALK ABOUT SOMETHING MOMENTARILY. BEFORE I DO THAT, I WANT TO SEE IF ANYBODY HAS ANY QUESTIONS ON ANY OF THE CHARTS THAT I'VE TALKED ABOUT SO FAR. QUESTION? ACTUALLY, HOLD ON ONE SECOND. LET HER GIVE YOU THE PHONE. HI. DO YOU DO ANY TRENDING ANALYSIS OF YEAR-TO-YEAR? YES. BECAUSE I SAW WITH HIRING STUFF AND TURNOVER, IT SEEMED LIKE WITH COVID, OF COURSE, SOME OF THESE NUMBERS MIGHT CHANGE BECAUSE REMOTE WORK BECOMES MORE PREVALENT AND TURNOVER AND JUST THE GENERAL AVAILABILITY OF ROLES OUTSIDE OF SOMEONE'S IMMEDIATE AREA BECAME A LOT MORE AVAILABLE. AND I FIGURED THAT MIGHT HAVE A SAY IN YOU KNOW, REASONS PEOPLE ARE LEAVING. SO I'M JUST CURIOUS IF YOU HAVE THAT. YEAH, EXACTLY. SO TWO THINGS ON THE TREND ANALYSIS. ONE IS, YES, I DO. THE TREND AROUND THE QUESTION OF RETENTION, --- Page 11 --- THAT MONEY, I THINK THAT THE FIRST YEAR THAT IT WAS NOT THE TOP WAS 21, IF I REMEMBER CORRECTLY. BUT THE OTHER REASON WHY I'VE BEEN SORT OF REALLY PUSHING TO RELEASE THE INFORMATION, THE DE-IDENTIFIED RESPONSES IS, AND THIS IS WHAT I'M GOING TO GO INTO NEXT, IS YOU CAN GET THE DATA AND START TO DO SOME OF THAT IF YOU'RE INTERESTED. WHAT I RELEASE, BEFORE I START TO GO INTO THE NEXT THING, ANY OTHER QUESTIONS? ANYTHING ELSE? YEAH, OKAY. GET A COUPLE OF THEM. BECAUSE I'M GOING TO GO INTO THE JupyterLab notebook stuff, WHICH I'LL THEN SHOW YOU HOW YOU CAN LOOK AT SOME OF THAT TOO BECAUSE THERE'S USUALLY QUESTIONS LIKE THAT FROM PEOPLE SPECIFICALLY. PLEASE. HEY, SO FOR THE OUTSOURCING SLIDE THAT YOU HAD ON THERE, YOU HAD THINGS LIKE THREAT RESEARCH AND THREAT HUNTING. BUT THEN ON THE HIRING SKILLS THAT PEOPLE WANTED WAS LIKE THREAT ANALYSIS AND INTELLIGENCE ANALYSIS. SO WE HAVE A LOT OF OUTSOURCING FOR THINGS THAT PEOPLE ARE LOOKING FOR TO GET, THEY WANT THEIR STAFF TO BE INTELLIGENT ON. IS IT JUST THEY'RE NOT FINDING THESE PEOPLE? YEAH, THAT'S A GREAT QUESTION. IT'S ACTUALLY A REALLY INTERESTING QUESTION. IN TERMS OF THE INSIGHT OF, OKAY, WELL, WE'RE OUTSOURCING THAT A LOT, BUT YOU'RE ALSO SAYING THAT A SKILL SET THAT MANAGERS ARE LOOKING FOR IS THREAT ANALYSIS, RIGHT? SO THE FIRST PART OF THIS IS THAT A LOT OF ORGANIZATIONS ACTUALLY DO STUFF, AND I'M GOING TO PUT THIS IN HERE IN TERMS OF SOME OF THE ACTIVITIES THAT ARE IN HERE AND TRY TO FIND WHERE THERE IS. SO THREAT HUNTING, THREAT RESEARCH, VULNERABILITY ASSESSMENT. A LOT OF ORGANIZATIONS ARE ACTUALLY DOING BOTH INTERNAL AND EXTERNAL. AND SO THERE'S SOME MORE GRANULAR QUESTIONS AROUND THE THREAT INTELLIGENCE SPECIFICALLY, AS AN EXAMPLE, WHERE IT'S A QUESTION OF ARE YOU DOING INGESTION OF IOCs OR ARE YOU DOING THE THREAT INTELLIGENCE ORIGINATION WHERE YOU'RE ACTUALLY DOING THE ANALYSIS AND THEN SHARING THAT. SO I THINK THAT THAT'S PART OF THE ANSWER TO YOUR QUESTION IS THAT WHAT THREAT INTELLIGENCE IS ACTUALLY TAKES. A VARIETY OF FORMS. THE OTHER PART OF IT IS I THINK THAT THREAT INTELL IS GETTING OUTSOURCED A LOT BECAUSE IT'S LIKE, YEAH, JUST GO TO THE DARK WEB, PULL ALL THE STUFF THAT MIGHT BE ABOUT MY COMPANY AND SEND IT TO ME. BUT THEN INTERNALLY WE NEED SOMEBODY WHO UNDERSTANDS, OKAY, NOW I'VE GOT THIS, YOU KNOW, ENORMOUS LIST OR STICK TAXI OR WHATEVER FORM THEY'RE GETTING IT IN. THAT NEEDS TO BE TURNED INTO A PRODUCT THAT BECOMES USEFUL TO THE EXECUTIVES, TO THE INTERNAL DETECTION ENGINEERING, MONITORING AND SO ON. SO THEN THEY HAVE A PERSON WHO'S TAKING THIS SORT OF GLUT OF DATA THAT'S COMING FROM THEIR THIRD PARTY OUTSOURCED INTELLIGENCE PROVIDER IN ORDER TO MAKE THAT USEFUL. SO THAT'S HOW I THINK THAT THAT'S PLAYING OUT. THE OTHER PART OF IT IS THREAT ANALYSIS ALSO BECOMES A RISK ANALYSIS PERSPECTIVE. SO IF YOU CAN UNDERSTAND THE THREAT, --- Page 12 --- THEN YOU CAN MAKE IT UNDERSTANDABLE TO THE ORGANIZATION ON HOW SHOULD WE RESPOND TO THAT. RIGHT? BUT A GREAT AND VERY INSIGHTFUL QUESTION. HAVE YOU HAD A HARD TIME WITH PEOPLE SELF-IDENTIFYING AS WORKING IN THE SOCK? FOR EXAMPLE, I KNOW A LOT OF PEOPLE WHO ARE INSTANT RESPONDERS WHO ARE VERY FIRM. THEY'RE LIKE, I DO NOT WORK IN THE SOCK. I AM INSTANT RESPONDER. OR PEOPLE IN THREAT INTEL I KNOW ALSO TAKE THAT LINE. YEAH. SO THAT'S ALSO A GOOD QUESTION. FOR ME, THE WAY THAT I'VE TRIED TO DO IT IS, LOOK, IT DOESN'T REALLY MATTER IF YOU'RE IN THE SOCK. YOU'RE IN THE SOCK. IF YOU CALL IT A SOCK, IF YOU CALL IT A FUSION CENTER, IF YOU DO INSTANT RESPONSE, THAT'S FINE. AND THERE IS ESSENTIALLY ALMOST A STIGMA ASSOCIATED WITH THE NOTION OF THE MONITORING COMPONENT OF IT. SO, YES, I'VE DEFINITELY OBSERVED THAT, THAT A LOT OF PEOPLE PUSH OFF THAT LABEL OF THE SOCK BECAUSE FOR SOME REASON IT'S PERCEIVED AS NEGATIVE. AND I THINK ONE OF THE REASONS WHY IT'S PERCEIVED AS NEGATIVE IS THERE'S THIS LABEL, AND IT'S L1. NO ONE WANTS TO BE CONSTRUED AS L1 BECAUSE THAT'S LIKE THE ENTRY LEVEL CYBER JOB. AND THERE WAS A TALK, SO I'VE DONE, YOU KNOW, DIFFERENT PRESENTATIONS FOR YEARS, AND ONE OF THE ONES THAT I DID IN 2018, IT WAS A SOCK SUMMIT THAT I WAS DOING THROUGH SANS. I'M GOING TO GET HIS LAST NAME WRONG, BUT ISMAIL CAITANO, I THINK WAS HIS LAST NAME. HE GAVE A PRESENTATION AT THE TIME. HE WAS THE VERIZON MSSP MANAGER. AND I'M POINTING, YOU KNOW, OUT TOWARD DULLESS, RIGHT? SO IT'S LIKE WHAT HE SAID WAS, YOU KNOW, IT'S A REALLY INTERESTING THING THAT THE MOST IMPORTANT DECISION IN A SOCK IS GIVEN TO THE MOST JUNIOR ROLE. AND USUALLY UNDER TREMENDOUS TIME PRESSURE, TEN MINUTE SERVICE LEVEL OBJECTIVE IN ORDER TO MAKE A DECISION, DO WE NEED TO ACT OR IS THIS A FALSE POSITIVE? AND HE SAID THAT WHAT THIS IS CREATING IS A STIGMA AND ESSENTIALLY AN OPPORTUNITY FOR MISSING A LOT OF STUFF. SO, YES, THE IDEA OF, LIKE, DO WE NOT LIKE THE SOCK? PEOPLE DON'T LIKE IT. AND IT'S KIND OF AN INTERESTING THING. I CHOSE THIS AREA INTENTIONALLY MANY YEARS AGO BECAUSE I FELT LIKE IT WAS AN INTERESTING THING. I FELT LIKE IT WAS A BIGGEST PROBLEM FOR MOST ORGANIZATIONS THAT THEY CANNOT FIGURE OUT HOW TO OPERATIONALIZE THEIR SECURITY CAPABILITY. AND IT PERSISTS IN THE WORLD TODAY. SO, YEAH, ANOTHER QUESTION. IF YOU TRY TO TAKE THIS MORE TO, LIKE, A MATURITY MODEL, SO, YOU KNOW, AS YOU SHOWED THE STATISTICS OF WHAT GOING THROUGH, OKAY, IF THEY'RE OUTSOURCING MORE, IT'S NOT NECESSARILY A BAD THING, LIKE THE SSP DAYS. THEY'RE MORE MOVING TOWARD AN MDR, XDR, NEXT GENERATIONS, ALL THAT TYPE OF STUFF, RIGHT? SO IS THERE A WAY TO KIND OF GAUGE THE JOURNEY THAT EVERYBODY'S AT THE SAME LEVEL, RIGHT? --- Page 13 --- NEW SPEAKER Yeah, that's an awesome question. I HAVE A BUNCH OF OTHER TALKS THAT I DO ABOUT MATURITY, BUT LET ME ANSWER YOUR QUESTION. THERE IS A MATURITY BASIS THAT I USE WHEN I DO MATURITY ASSESSMENTS FOR ORGANIZATIONS. OUT OF THE NETHERLANDS, IF YOU LOOK UP SOC-CMM, S-O-C-C-M-M, THERE'S ACTUALLY A BASIS FOR DOING A MATURITY ASSESSMENT, AND WHAT HE'S DONE OVER THE LAST SEVERAL YEARS IS TO TRY TO DEVELOP A CONSISTENT ASSESSMENT CRITERIA, AND I'M WORKING WITH HIM, AND HE HAS NOW THIS NETWORK OF PEOPLE WHO ARE ACTUALLY DOING MATURITY ASSESSMENTS TO PROVIDE A BASIS OF COMPARISON BETWEEN SOCS. SO THIS SOC SURVEY IS KIND OF A DIFFERENT THING OF LIKE TRENDING AND WHAT PEOPLE ARE DOING AND A VIEW OF IT, BUT WHAT YOU'RE TALKING ABOUT IS ABSOLUTELY WE WANT TO HAVE A GLOBAL COMPARATIVE INDEX ON WHAT IT MEANS TO BE A SECURITY OPERATION CENTER. AND YOU MENTIONED ANOTHER THING ABOUT THE NEGATIVE PERCEPTION OF OUTSOURCING. I THINK OUTSOURCING IS PHENOMENAL. I THINK THERE'S A VERY HIGH VALUE PROPOSITION TO IT. I ALSO THINK THAT THE ONE THAT I JUST HAD UP THERE A MOMENT AGO IN TERMS OF INTERNAL ONLY, OUTSOURCED ONLY, AND THEN BOTH, I THINK THAT THE CAREFUL SELECTION OF VENDOR PARTNERS WHO CAN SUPPLEMENT YOUR CAPABILITY AND DRIVE YOUR STAFF TO DO BETTER IS REALLY THE RIGHT MIX. AND PENTESTING IS A GREAT EXAMPLE OF THIS. IF YOU CAN GET AN EXTERNAL PARTY WHO HAS EXPERTISE TO COME IN AND DO ONE PENTEST, AND YOU'RE NOT ABSORBING THE OPERATIONAL ACTIVITY INTO YOUR INTERNAL PENTEST TEAM, YOU'RE MISSING A HUGE OPPORTUNITY TO BE ABLE TO REUSE THAT CAPABILITY. THIS IS SOMETHING THAT, AS I WORK WITH ORGANIZATIONS TO HELP THEM TO REFACTOR HOW THEY'RE DOING IT, IT'S SIMPLY A MATTER OF IF YOU'RE GETTING A REPORT THAT TELLS HOW THE THIRD PARTY DID IT, YOUR STAFF SHOULD BE ABLE TO REPLICATE THAT. YOU SHOULDN'T HAVE TO GO BACK AND PAY FOR THAT AGAIN. WHAT YOU SHOULD BE PAYING FOR IN YOUR THIRD PARTY PENTEST IS CREATIVITY AND GROWTH AND THEIR RESEARCH AND THEIR EXPERTISE. AND IF YOU'RE NOT CAPTURING THAT OPERATIONALLY INTERNALLY, YOU'RE MISSING AN OPPORTUNITY. YOU'RE BEING WASTEFUL WITH YOUR RESOURCES. LET ME GO INTO THIS, AND I'M GOING TO KIND OF DO THIS FAST, AND THEN I'LL TAKE ONE MORE ROUND OF QUESTIONS AFTER THAT. THE OTHER THING, AND THIS IS KIND OF LIKE THE GENTLEMAN'S QUESTION HERE EARLIER, IS I THINK IT WAS IN 2018, THE FIRST TIME THAT SOMEBODY CONTACTED ME AFTER I RELEASED THE SURVEY, AND THIS PERSON ASKED ME, HEY, YOUR REPORT IS USEFUL. I WORK IN AN ORGANIZATION THAT HAS 100,000 EMPLOYEES. WHAT I REALLY WANT TO KNOW IS WHAT THE OTHER, LIKE, TEN ORGANIZATIONS THAT ARE LIKE ME IN THE FINANCIAL INDUSTRY THAT DO THIS THING. AND SO I COULDN'T GIVE HIM THE RESPONSES. I COULDN'T SHARE THAT. I KEPT SORT OF PUSHING IN ORDER TO GET IT TO BE RELEASED. NOW, IN 2020, I ACTUALLY RELEASED THE SURVEY MYSELF. --- Page 14 --- I DIDN'T DO IT WITH SANS INSTITUTE BECAUSE THEY DIDN'T DO THE SURVEY IN 2020. SO IN 2020 WAS THE FIRST YEAR THAT ALL OF THE RESPONSES, I MEAN, DE-IDENTIFIED, OF COURSE, GOT RELEASED, AND SINCE THEN THAT'S HAPPENED. SO LET ME JUST DO THE SETUP OF THE JUPITER NOTEBOOK AND TALK ABOUT SOME OF THE THINGS FOR THIS. IF YOU WANT TO DO THIS ON YOUR OWN, YOU NEED PYTHON, YOU NEED PANDAS, YOU NEED SEABORN AND JUPITERLAB. THESE ARE ALL THE LINKS TO BE ABLE TO GET THOSE THINGS. ALSO, HAVING THE EXCEL SUPPORT IS NECESSARY. I LIKE WORD CLOUDS. SO THERE'S SOME CAPABILITY TO DO THAT. AND THEN THIS IS HOW YOU WOULD START IT UP. . NOW, I ACTUALLY, I'M NOT GOING TO TYPE ANYTHING. I DO HAVE A SUBSET OF THE JUPITERLAB STUFF, SO I CAN JUST TALK THROUGH WHAT I BASICALLY DO IN MY ANALYSIS FOR THIS. YEAH, THAT'S A FRIEND OF MINE STICKER. SO THIS IS THE, NOT WHAT I WANTED. THIS IS THE JUPITER NOTEBOOK THAT I BASICALLY HAVE PULLED OUT A FEW THINGS OF WHAT I DO WHEN I DO THE ANALYSIS. I ALSO WORK WITH BARBARA FILKINS. AND SO SHE DOES A BUNCH OF ANALYSIS. I DO A BUNCH OF ANALYSIS. WE COMBINE IT. BUT THIS IS AVAILABLE FOR DOWNLOAD. AND THE IDENTIFIED DATA IS AVAILABLE FOR DOWNLOAD FROM THE SOC SURVEY. SO IF YOU WANT TO SAY, LIKE, OH, MY ORGANIZATION IS HEADQUARTERED IN ASIA, I ONLY WANT TO LOOK AT THE OTHER ORGANIZATIONS THAT SAY THAT THEY'RE HEADQUARTERED IN ASIA, YOU CAN DO THAT. . SO THE PRELIMINARY STUFF HERE IS JUST, JUST A BUNCH OF, LIKE, IS THIS WORKING PROPERLY? DO I HAVE THE INFORMATION IN MY DATA SET? THE NEXT THING, THIS IS SOMETHING THAT'S A BIG CHALLENGE, AND THIS IS KIND OF SOME EXPLORATORY ANALYSIS WHEN YOU'RE DEALING WITH DATA THAT YOU USUALLY NEED TO DO, IS THE WAY THAT THE SPREADSHEET IS ACTUALLY LISTED OUT, THE LABELS ARE TERRIBLE. AND THEY'RE REALLY DIFFICULT TO COMPREHEND. SO WHAT I END UP DOING IS RELABELING EVERYTHING AND I JUST KEEP REUSING THE CODE IN ORDER TO DO THAT, AND I ACTUALLY TAKE THE ORIGINAL QUESTION AND I WRITE A LITTLE PYTHON, I ELIMINATED IT JUST FOR, YOU KNOW, TIGHTENESS IN THIS PARTICULAR NOTEBOOK, AND THEN I USE THE QUESTION, WHICH IS WHAT THE COMMENT IS FOR EACH OF THESE, IN ORDER TO BE ABLE TO RELABEL ALL OF THE COLUMNS FOR ALL OF THE RESPONSES. NOW, ON THE ONE HAND, THIS IS RIDICULOUS TO SPEND THE TIME DOING. ON THE OTHER HAND, MY BRAIN CAN'T WRAP AROUND THE, LIKE, 500 COLUMNS OF DATA THAT'S ACTUALLY IN THERE. 500 COLUMNS OF RESPONSES THAT ARE ACTUALLY IN THERE. SO THIS IS JUST THE CODE FOR BASICALLY REMAPPING THAT IN A DATA FRAME INSIDE OF PANDAS TO BE ABLE TO WORK THROUGH THAT. THEN I HAVE CODE DOWN HERE THAT SAYS IF YOU WANT TO SHIFT IT TO, LIKE, THIS IS THE ORGANIZATION --- Page 15 --- SIZE THAT I WANT TO JUST LOOK AT BIG ORGANIZATIONS, BASED OUT OF ASIA. NOW, I RELEASED THIS BECAUSE I LITERALLY USE THIS WHEN I'M ADVISING COMPANIES. LIKE, I GO BACK AND WILL SAY, OKAY, JUST AS AN EXAMPLE, I HAD AN ORGANIZATION APPROACH ME TO HELP THEM TO PROVIDE GUIDANCE TO A CUSTOMER OF THEIRS THAT WAS BASED IN SOUTH AMERICA IN TERMS OF HOW TO DO SOME WORK, AND I WAS HELPING THEM TO MAKE DECISIONS, AND I WAS LIKE, OKAY, THIS IS HOW I WOULD DO THIS WHEN I GO INTO THE SURVEY DATA AND DO THESE PIVOTS TO BE ABLE TO ANSWER THE QUESTIONS FOR A COMPARATIVE ANALYSIS. I JUST MOVED IT BACK UP. OKAY. SO THERE'S SOME STUFF IN HERE IN TERMS OF HOW TO DO REGIONAL SELECTIONS OF THE CODES IN THERE IN ORDER TO CREATE ALTERNATE DATA FRAMES TO WORK THROUGH THAT. THEN, AGAIN, FROM SORT OF AN EXPLORATORY PERSPECTIVE, I DO A BUNCH OF GRAPHING JUST TO KIND OF WRAP MY BRAIN AROUND WHAT I'M DEALING WITH. SO I HAVE SOME CODE THAT BASICALLY WORKS THROUGH ALL OF THE DIFFERENT GRAPHS THAT ARE THERE. SO THIS JUST TALKS ABOUT THE DIFFERENT DETAILS OF THE TITLES OF HOW I WANT THE LABEL ON THE CHARTING TO COME OUT, SO THAT'S ALL IN THERE. AND THEN I WRITE IT WHERE I BASICALLY DUMP ALL THE GRAPHS OUT TO THE FILE SYSTEM AS WELL AS PUTTING THEM INTO THIS. SO THIS IS, AGAIN, WHEN I'M DOING THE PRELIMINARY ANALYSIS, I GO THROUGH AND I CHART EVERY SINGLE QUESTION. AND THE OTHER REASON WHY I'M SHOWING THIS TO YOU IS ALSO WHEN YOU START DEALING WITH DATA RESPONSES, WHATEVER YOU'RE DEALING WITH, IT'S REALLY HARD TO THINK THAT YOU'RE, YOU KNOW, DOING IT APPROPRIATELY. SO IF YOU CAN SHARE YOUR METHODOLOGY OF ANALYSIS WITH OTHER PEOPLE AND EXPOSE YOUR THINKING, IT IS REALLY HELPFUL TO NUMBER ONE, MAKE SURE YOU'RE NOT MAKING MISTAKES. NUMBER TWO, THEN GIVE OTHER PEOPLE WHO WANT TO DO ANALYSIS AN OPPORTUNITY TO PEER INTO HOW YOU'RE THINKING ABOUT IT. AND I FOUND THAT JUPITER IS PHENOMENAL FOR THIS BECAUSE I CAN JUST SHARE THIS WITH YOU AND SAY, OKAY, LOOK AT IT. AND SO I'VE GOT A LOT OF DETAILS IN HERE, BUT BASICALLY I'M SETTING IT UP TO WHERE I'M TRYING TO MAKE GRAPHICS THAT ARE READY TO DROP INTO A REPORT. I WILL ALSO SAY INSIDE OF CYBERSECURITY OPERATIONS, WE'RE USUALLY REALLY TERRIBLE AT REPORTING AND OUR EXPLANATION OF OUR RATIONALE, AND SO I'M TRYING TO KIND OF PROVIDE A GOOD EXAMPLE OF HOW I APPROACH THAT THINKING AND THE RIGOR THAT I USE IN ORDER TO DO THAT IN ORDER TO BE ABLE TO MAKE ASSESSMENTS OF THINGS THAT I DON'T FULLY UNDERSTAND. SO THIS IS FOR ME, THIS IS KIND OF LIKE MY EXPLORATORY ANALYSIS, AND THERE'S SOME CODE WHERE I'M DEALING WITH THINGS WHERE I DON'T WANT CHARTS THAT HAVE 30 ITEMS IN IT, SO I FILTER --- Page 16 --- SOME STUFF DOWN. BUT I BASICALLY JUST AM ABLE TO VISUALIZE EVERY SINGLE QUESTION THAT COMES OUT, BUT THAT'S THERE. AND THEN I GO INTO SOME OF THE DETAILS OF HOW I DEAL WITH TECHNOLOGY. I THINK THIS IS WHAT I INCLUDED AS THE FINAL SORT OF DEMO COMPONENT INSIDE OF HERE. SO THESE ARE ALL STILL JUST GETTING IT TO WHERE I'VE GOT THE CHARTS THE WAY THAT I WANT. AND THEN IN THE TECHNOLOGY HERE, I TALK ABOUT SOME OF THE DIFFICULTY OF FINDING CERTAIN SEGMENTS OF DATA AND THEN PIVOTING AROUND INSIDE OF IT IN ORDER TO DO THE VISUALIZATIONS AND THE COUNTING AND THE RATIONALE OF HOW I BASICALLY DID THE GPA-BASED ASSESSMENT OF THE DIFFERENT TECHNOLOGIES. THIS IS SORT OF A FIRST STAB AT THE VIEW OF WHAT WERE PEOPLE'S OPINIONS OF THE TECHNOLOGY. AND IN THIS, AND OF COURSE, I'M NOT GOING TO TAKE THE TIME TO GO INTO IT. BUT WHEN I WAS FIRST LOOKING AT THIS, I WAS ASKING THE QUESTION, WHERE ARE THERE CASES WHERE PEOPLE LOVE IT AND HATE IT? BECAUSE I CAN HAVE ONE THING WHERE EVERYBODY GAVE IT A C AND ANOTHER ONE WHERE THERE ARE LOTS OF A'S AND F'S AND IT CAN END UP WITH THE SAME GPA. SO I WAS TRYING TO LOOK AT SOME OF THESE FROM A VISUALIZATION PERSPECTIVE AND YOU CAN DO IT STATISTICALLY AS WELL TO DECIDE ON THIS. AND THEN JUST SOME DETAILS WORKING THROUGH THE NUMBERS ON WHETHER PEOPLE LIKED IT OR NOT. AND THEN ENDING UP WITH THE GPA'S. SO THIS JUBERLAB NOTEBOOK, IT'S ON THAT WEBSITE THAT YOU CAN DOWNLOAD IF YOU DO HAVE ANALYSIS THAT YOU WANT TO DO. IF YOU DO HAVE OTHER QUESTIONS, FEEL FREE TO ASK. NOW THAT I'VE SHOWN YOU THIS, IT MAY SORT OF TRIGGER SOME OTHER THOUGHTS. ANYTHING ELSE? GO AHEAD. ARE WE OUT OF TIME? YOU CAN ASK ME AFTERWARDS. OKAY. NEW SPEAKER SO LET'S GIVE A HAND TO CHRISTOPHER CROWLEY. THANK YOU.

Related Q&A

  • Q: Overview
    A: Presentation details for 2023 SANS SOC Survey Review