Python Threat Hunting Tools: Part 12 – MISP and CrowdStrike Falcon Integration

Welcome back to this series on building threat hunting tools. In this series, I will be showcasing a variety of threat hunting tools that you can use to hunt for threats, automate tedious processes, and extend to create your own toolkit!

Most of these tools will be simple, focusing on being easy to understand and implement. This is so that you, the reader, can learn from these tools and begin to develop your own. There will be no cookie-cutter tutorial on programming fundamentals like data types, control structures, etc. This series will focus on the practical implementation of scripting through small projects.

You are encouraged to play with these scripts, figure out ways to break or extend them, and try to improve their basic design to fit your needs. I find this the best way to learn any new programming language/concept and, certainly, the best way to derive value!

In this installment, you will learn how to automatically export Indicators of Compromise (IOCs) for your MISP instance and upload them into the Endpoint Detection and Response (EDR) solution CrowdStrike Falcon.

Let’s first discover what MISP and CrowdStrike Falcon are.

What is MISP?

MISP (Malware Information Sharing Platform and Threat Sharing) is an open-source threat intelligence platform that allows you to share, collate, analyze, and distribute threat intelligence. It is used by finance, healthcare, telecommunications, government, and technology organizations to share and analyze information about the latest threats. Security researchers, threat intelligence teams, incident responders, and the wider cyber security community all use MISP to collaborate in their defensive efforts.

The platform provides a structured and standardized framework for collecting, storing, and sharing threat intelligence data, enabling collaboration and enhanced defense against cyber threats. It has mappings with existing threat intelligence frameworks (e.g., MITRE ATT&CK, CAPEC, etc.) and strong integrations with security products (e.g., CrowdStrike Falcon, Intel471, etc.). MISP is the defacto open-source threat intelligence platform mature organizations use to track threats and collaborate.

You can learn more about MISP in my Threat Intelligence with MISP series. This series teaches you everything you need to know to use the platform and build your MISP instance to gather, analyze, and share threat intelligence. Start with Threat Intelligence with MISP: Part 1 — What is MISP?

What is CrowdStrike Falcon?

CrowdStrike Falcon is a cloud-native Endpoint Detection and Response (EDR) tool that is designed to protect an organization’s endpoint devices from various cyber threats, such as malware, ransomware, and threat actors. The tool uses threat intelligence, behavior analytics, and machine learning to adapt to the latest vulnerabilities and exploits. 

CrowdStrike Falcon has become the industry leader in EDR over the last several years, boasting 100% protection, 100% Protection, 100% Visibility, and 100% Detection in MITRE Engenuity ATT&CK. The tool can also be extended through various partner applications in the CrowdStrike marketplace to provide a complete solution tailored to your organization’s needs.

The CrowdStrike Falon platform is the defacto choice for many organizations who want a complete EDR solution, and you are likely to come across it as a cyber security professional. This is why this article chose it as the EDR tool to demo (along with it having a robust API). You can try CrowdStrike Falcon for free on a 15-day trial if you don’t have access to the tool.

The Problem

As a threat intelligence analyst, you need a way to consume, analyze, and distribute threat intelligence to aid your organization in combating the latest cyber threats. If you have been following along with the Threat Intelligence with MISP series, you will have seen how you can do this using the MISP platform. However, speed is king in security operations, and you need a way to distribute operational intelligence to your front-line defenders quickly.

You need to take the Indicators of Compromise (IOCs) you have in your MISP instance and distribute them automatically to your security solutions so they can detect if an IOC is present in your environment and alert a member of your security operations team. How can you do this with MISP and CrowdStrike Falcon? 

The Solution

Fortunately for you, MISP and CrowdStrike Falcon have Application Programming Interfaces (APIs) you can use to programmatically export IOCs from MISP and upload them to CrowdStrike Falcon for detection. You just need to create a Python script connecting the two technologies. Let’s do that!

You can learn how to use the MISP API in Threat Intelligence with MISP Part 6 – Using the API, if you want to know how to export IOCs from MISP, you can read Threat Intelligence with MISP Part 7 – Exporting IOCs.

Python Modules and API Variables

To begin, you need to import several Python modules that will help you interact with the MISP and CrowdStrike Falcon APIs, validate IOCs, create dates, and display output:

# to interact with CrowdStrike Falcon via Python
from falconpy import IOC

# to interact with MISP API
from pymisp import PyMISP

# for miscellaneous script tasks
from pprint import pprint
from validators import ip_address
from datetime import datetime, timedelta

Next, you want to define your API configuration variables that allow you to connect and authenticate to the MISP and CrowdStrike Falcon APIs.

To do this, I have used a Python data class named config that holds all my API configuration variables. I have done this to keep sensitive data (API configuration) separate from my script and any version control system using a .gitignore file.

# the config dataclass contains API configuration variables, it is in a module called config
from config import config

# MISP authentication vars
MISP_URL = config.MISP_URL
MISP_KEY = config.MISP_KEY
MISP_VERIFYCERT = config.MISP_VERIFYCERT

# CrowdStrike authentication vars
CLIENT_ID = config.CS_CLIENT_ID
CLIENT_SECRET = config.CS_CLIENT_SECRET

The start of your script should look similar to this.

Now you are ready to start on your MISP to CrowdStrike IOC script.

Getting IOCs from MISP

First, you need to gather all of your MISP attributes and transform them into IOCs that the CrowdStrike Falcon tool can use. To do this, you can create a function that takes in your MISP API authentication variables and returns a Python dictionary containing IOCs CrowdStrike can use. The keys are the IOC type and the values are a list of IOCs of that specific IOC type. 

To start, create the function signature, authenticate to the MISP API (using your API configuration variables), and gather all the MISP attributes that are on the MISP instance.

def GetMispAttributes(misp_url, misp_key, misp_verifycert):
    # authenticate to misp 
    misp = PyMISP(misp_url, misp_key, misp_verifycert, debug=False)

    # get all IOCs with IDS flag set to true and published in last 89 days
    attributes = misp.search(controller='attributes', to_ids=1, pythonify=True, publish_timestamp='89d')

Here, I have only collected the MISP attributes that can be used with Intrusion Detection Systems (IDS) and were published in the last 90 days. These have been stored in the attributes variable (a custom Python object created by the MISP Python module). 

Now you need to go through this attributes object and extract only the IOCs that CrowdStrike Falcon can use. These are domains, IPv4 and IPv6 addresses, and SHA256 and MD5 file hashes. To do this, you can create Python lists that represent IOC buckets. You can then loop through the attributes object, check the IOC type, and then sort them into each IOC bucket.

# add IOCs to bucket
ipv4 = []
ipv6 = []
domain = []
url = []
hostname = []
sha256 = []
md5 = []
sha1 = []
other = []

for i in attributes:
    if (i.type == "ip-dst"):
        # check if IPv4 or IPv6
        if (ip_address.ipv4(i.value)):
            ipv4.append(i.value)
        elif (ip_address.ipv6(i.value)):
            ipv6.append(i.value)
        else:
            other.append(i.value)
    elif (i.type == "ip-dst|port"):
        addr = ipv4.append(i.value.split('|')[0])
        # check if IPv4 or IPv6
        if (ip_address.ipv4(addr)):
            ipv4.append(addr)
        elif (ip_address.ipv6(addr)):
            ipv6.append(addr)
        else:
            other.append(addr)
    elif (i.type == "domain"):
        domain.append(i.value)
    elif (i.type == "domain|ip"):
        # split domain an ip, append to respective lists
        domain.append(i.value.split('|')[0])
        addr = ipv4.append(i.value.split('|')[1])
        # check if IPv4 or IPv6
        if (ip_address.ipv4(addr)):
            ipv4.append(addr)
        elif (ip_address.ipv6(addr)):
            ipv6.append(addr)
        else:
            other.append(addr)
    elif (i.type == "url"):
        url.append(i.value)
    elif (i.type == "hostname"):
        hostname.append(i.value)
    elif (i.type == "hostname|port"):
        # split hostand and port
        hostname.append(i.value.split('|')[0])
    elif (i.type == "sha256"):
        sha256.append(i.value)
    elif (i.type == "filename|sha256"):
        # split filename and hash, append hash to sha256 list
        sha256.append(i.value.split('|')[1])
    elif (i.type == "md5"):
        md5.append(i.value)
    elif (i.type == "filename|md5"):
        # split filename and hash, append hash to md5 list
        md5.append(i.value.split('|')[1])
    elif (i.type == "sha1"):
        sha1.append(i.value)
    elif (i.type == "filename|sha1"):
        # split filename and hash, append hash to sha1 list
        sha1.append(i.value.split('|')[1])
    else:
        other.append(i.value)

Once you have sorted your IOCs into buckets, you can add some reporting to the function to provide the user of your script a breakdown of the IOCs pulled from their MISP instance. 

ipv4_length = len(ipv4)
ipv6_length = len(ipv6)
domain_length = len(domain)
url_length = len(url)
hostname_length = len(hostname)
sha256_length = len(sha256)
sha1_length = len(sha1)
md5_length = len(md5)


# print totals
print(f"[+] Total MISP indicators: {ipv4_length + ipv6_length + domain_length + url_length + hostname_length + sha256_length + sha1_length + md5_length}")
print(f"+++ Network Indicators +++ ")
print(f"- IPv4 addresses: {ipv4_length}")
print(f"- IPv6 addresses: {ipv6_length}")
print(f"- Domains: {domain_length}")
print(f"- URLs: {url_length}")
print(f"- Hostnames: {hostname_length}")
print(f"+++ Endpoint Indicators +++")
print(f"- SHA256 hashes: {sha256_length}")
print(f"- SHA1 hashes: {sha1_length}")
print(f"- MD5 hashes: {md5_length}")
print(f"[+] Total \"other\" IOCs: {len(other)}")
print(f"[+] Total indicators to upload to CrowdStrike: {ipv4_length + ipv6_length + sha256_length + md5_length + domain_length}")

Finally, you can return only the IOC buckets that CrowdStrike Falcon can utilize. These are returned in a Python dictionary containing the IOC type (key) and a list of IOCs (value).

# return indicators list (tuple of type and value)
cs_indicators = {
    "ipv4": ipv4,
    "ipv6": ipv6,
    "domain": domain,
    "sha256": sha256,
    "md5": md5,
}
return cs_indicators

The whole function looks like this.

Uploading IOCs to CrowdStrike Falcon

With your CrowdStrike Falcon IOCs in hand, you can now create a function to upload them to your CrowdStrike Falcon instance using the EDR tool’s API. First, create your Python function signature and authenticate to the CrowdStrike Falcon API using the global API configuration variables you previously created. Your function will take your IOC dictionary returned by the GetMispAttributes() function as a parameter.

def UploadIOCs(iocs_to_upload):
    # authenticate to CrowdStrike Falcon
    falcon = IOC(client_id=CLIENT_ID, client_secret=CLIENT_SECRET)

Next, you must define some IOC details to add to the IOC value you upload to CrowdStrike Falon. These include the platforms on which the IOC will be detected and a timestamp for when the IOC should expire. In addition, I have added some code to display information to the user to indicate how many IOCs the script will try to upload to CrowdStrike Falcon and to keep track of successful IOC uploads using two lists.

# get count of iocs to upload 
print()
count = 0
for i in iocs_to_upload:
    count += len(iocs_to_upload[i])
print(f"[+] Uploading {count} MISP indicators.")

# upload indicators
ioc_platforms = ["Mac", "Windows", "Linux"]
# UTC formatted date string (current date + 90 days)
now = datetime.now() + timedelta(days=90)
ioc_expiry_date = now.isoformat() + "Z" 
# lists to track successful IOC uploads
uploaded_iocs = []
failed_iocs = []

Now you can loop through the IOC types in the Python dictionary and create an inner for loop to go through each IOC in the IOC type lists. The inner for loop contains the CrowdStrike Falcon API call that adds the IOC to the EDR platform. It uses the indicator_create() method of the IOC object you previously created, along with several IOC attributes; action to take on detection, the IOC value, the IOC type, alert severity, platforms to detect on, a description, and an expiration date.

for ioc_type in iocs_to_upload:
  for ioc in iocs_to_upload[ioc_type]:
    response = falcon.indicator_create(action="detect", value=ioc, type=ioc_type, 
                  severity="high", platforms=ioc_platforms, applied_globally=True, 
                  retrodetects=True, description="IOC from MISP Database", expiration=ioc_expiry_date)

The retrodetects option seen in the code above will even go back in time to trigger detections if an IOC has been seen in the past. By default, CrowdStrike retains 90 days of logs.

After you have made your API call, you need to check its response. The response variable will contain a JSON object you can interact with, similar to a Python dictionary. The status_code indicates the success or failure of the API call. If successful, the IOC is added to the uploaded_iocs list for tracking. If the API call fails, the failure message is parsed out, printed to the terminal, and the IOC is added to the failed_iocs list for tracking.

if (response['status_code'] == 201):
    uploaded_iocs.append(ioc)
elif (response['status_code'] == 400):
    try: 
        print("[FAIL] - " + response['body']['resources'][0]['message'])
        failed_iocs.append(ioc)
    except:
        print(response)

Finally, the UploadIOCs() function prints information about the number of IOCs uploaded and how many IOCs are now in the CrowdStrike Falcon tool and are being detected.

pprint(f"[+] upladed {len(uploaded_iocs)} new IOCs to CrowdStrike Falcon IOC Management")
pprint(f"[+] failed to upladed {len(failed_iocs)} to CrowdStrike Falcon IOC Management (duplicates)")

# check new CrowdStrike indicator count
response2 = falcon.indicator_combined()
total_iocs = response2['body']['meta']['pagination']['total']
pprint(f"[+] There are now {total_iocs} CrowdStrike indicators.")

The function should look like this.

Tying Everything Together

Now you have a function for exporting IOCs from MISP and uploading them to CrowdStrike Falcon, you can pull everything together in the main function of your code. Print formatting messages to the terminal, call both functions, and you are done!

if __name__ == "__main__":
    print("--- script start ---")
    # connect and authenticate to CrowdStrike
    faclon = IOC(client_id=CLIENT_ID, client_secret=CLIENT_SECRET)

    # --- Step 1: Exports all attributes from MISP --- #
    indicators = GetMispAttributes(MISP_URL, MISP_KEY, MISP_VERIFYCERT)

    # --- Step 2:  Use the CrowdStrike API to upload these IOCs as indicators to CrowdStrike --- #
    UploadIOCs(indicators)

    pprint("--- script complete ---")

You can find the complete script as misp-to-crowdstrike.py on my GitHub page. Your main function should look like this.

Conclusion

You now have a fully functioning MISP to CrowdStrike IOC script that automatically distributes operational intelligence to your security operations team! 

You learned how to export attributes from MISP, transform them into IOCs the CrowdStrike Falcon EDR tool could utilize, and upload them to Falcon using the CrowdStrike API. This automation saves you from two web GUIs, manually parsing data, and a lot of time, considering this flow is part of your daily operations.

That said, this script is not perfect. There are areas of improvement and different ways of doing things that might suit your specific team’s operations better. For instance, your team might not use CrowdStrike Falcon and need to integrate with another API; your team might want to export IOCs from another threat intelligence database; you might just want to implement this script using an object-oriented programming style. The choices are yours. Give it all a go, and let me know what you come up with!

The next installment in this series will show you how to use the community version of VirusTotal to check if an IOC is suspicious or malicious programmatically. No need for web browser automation this time.  Till then, happy hunting!

Discover more in the Python Threat Hunting Tools series!