Evaluation of Local Security Event Management System vs. Standard Antivirus Software

Featured Application

This work can be applied to develop new anti-malware strategies based on event analysis.

Abstract

The detection and classification of threats in computer systems has been one of the main problems researched in Cybersecurity. As technology evolves, the tactics employed by adversaries have also become more sophisticated to evade detection systems. In consequence, systems that previously detected and classified those threats are now outdated. This paper proposes a detection system based on the analysis of events and matching the risk level with the MITRE ATT&CK matrix and Cyber Kill Chain. Extensive testing of attacks, using nine malware codes and applying three different obfuscation techniques, was performed. Each malicious code was analyzed using the proposed event management system and also executed in a controlled environment to examine if commercial malware detection systems (antivirus) were successful. The results show that evading techniques such as obfuscation and in-memory extraction of malicious payloads, impose unexpected difficulties to standard antivirus software.

1. Introduction

Antivirus software is the most common tool being used to protect the user’s computer from malware attacks. There are other protections methods such as local firewalls and corporate firewalls that protect the end-user from being totally exposed to attackers or malware already running on the local network. However, there are too many ways in which a piece of malicious software can get access to a personal computer. All sophisticated software, such as the operating system, programs running on a personal computer, and even software running on firewalls or other hardware equipment, contains bugs or flaws that may cause these systems to act in unexpected ways. Although most bugs are not significant on their own and do not pose a security risk, an attacker can take advantage of certain bugs to write programs called exploits to increase their arsenal of attacking tools. By combining a series of exploits into an “exploit chain”, it is possible to circumvent all the defenses one-by-one until reaching the victim’s computer to execute some kind of malicious software there. The fact that malware eventually reaches the end-user computer highlights the importance of malware detection software running locally.

This paper proposes a detection strategy based on events generated by the system and its corresponding mapping with Cyber Kill Chain and MITRE ATT&CK. The contributions of this paper are:

• A framework has been implemented to evaluate threats based on the analysis of events and subsequent classification in the Cyber Kill Chain and MITRE ATT&CK models.
• Using different obfuscation techniques, a set of malware samples has been built to evaluate the effectiveness of commercial antivirus systems and determine their detection and classification capabilities while dealing with obfuscated files.

For a better understanding of the contents of this paper, it was organized as follows: Section 2 describes all the elements that must be known to understand the methodology proposed in this paper; therefore, the classification of malware families, the current functioning of antivirus systems, the most common evasion techniques, and the threat classification model used to analyze events are explained. Section 3 describes the proposed methodology and its implementation. Section 4 describes the experiments performed and the results obtained and includes the comparison with several known antivirus systems. Section 5 analyzes the results obtained with the proposed methodology. Section 6 concludes this paper and presents future works.

2. Review and Background Knowledge

The detection and classification of threats [1] in networks and computer systems is an essential component for organizations. The objective is to be able to detect malicious programs (malware) that have evaded protection layers being able to reach a server or a user system. The emergence of these malicious programs originates back to 1971, when Robert Thomas created the first computer virus called “Creeper” [2,3]. Since then, the evolution in number of attacks and sophistication level has been astonishing, with different types of malwares appearing for all known computer systems.

2.1. Malware Family Classification

However, malicious code has not only evolved technically, but also in terms of threat. They not only damage files or computer systems, as they used to behave in the 1970s and 1980s, but they are also used to spy, steal information, or demand monetary ransoms.

Viruses: Usually incorporated into existing software, they are activated when the user executes the software. Their consequences are diverse, ranging from slowing down the system to corrupting or deleting information [4,5].

Worms: Do not need, at first, user intervention and have the ability, in contrast to computer viruses, to self-propagate through the network [6,7].

Trojans: Hidden inside apparently legitimate software for the purpose of going undetected. This type of malicious program can perform any action it has been programmed to perform on the system [8,9].

Spyware: Like worms, it does not require user intervention to install itself on the system and usually works in a hidden mode by collecting user or system information in an unauthorized manner [8,9].

Adware: Usually inserted in the system through the installation process of other software and its mission is to display unwanted advertising, usually in the form of pop-up windows without the user’s authorization. Now, there is Adware in the form of browser extensions or plugins.

Ransomware: It is generally executed from another malware, such as a worm, virus or trojan. Its mission is to completely sequester the system, encrypting all the files and demanding a payment from the user or organization [10,11,12,13].

Rootkit: It has the function to access and hide in particularly sensitive areas of the devices it infects, including areas that are not normally accessible to users. Their purpose is to take control of the system and facilitate its remote control while remaining hidden [14,15].

2.2. Actual Antivirus Threat Detection and Classification Systems

The main detection and classification software for this type of malware is the antivirus software, that have evolved in techniques, using mechanisms based on signature, heuristics, rules, and currently the use of artificial intelligence [16,17].

Signature detection [18,19,20] is the traditional method used by antivirus systems and is based on a database generated by the vendor. Any file downloaded to the system is compared to the database and if there is a match, it is malware. The problem with signature-based detection is that it will only detect those samples that have been previously identified and their signature is stored in the antivirus system database.

To complement signature-based detection and provide a solution to its limitations, heuristic detection techniques were developed [21,22,23]. The operation of heuristic algorithms is based on different criteria, each of them with a score, which determine whether a file is malicious or not. The most common three ways of performing this analysis are [24,25]:

Generic: Compares the actions of a file with another already identified as malicious.

Passive: Analyzes the file individually and tries to determine how it works.

Active: Runs the sample in a safe environment (sandbox) and determines if its activity is malicious. This strategy is difficult to implement, imposes significant delays, and it can be avoided by implementing payload activation delays.

Heuristic analysis has two fundamental problems, the first is the large number of false negatives and the second is the workload on the system. However, it improves the ability of antivirus software to detect new samples of malware.

Recently, to increase the capabilities of antivirus systems, the use of machine learning algorithms using artificial intelligence (AI) has been introduced [26,27,28]. The inclusion of these techniques allows for a large-scale data analysis, the identification of patterns and trends, as well as the automatic and rapid formulation of predictions. These systems are called Endpoint Detection and Response (EDR), also known as Endpoint Thread Detection and Response (ETDR), and implement and endpoint security mechanism [29] at the clients that collects data and sends it to a centralized console for processing, as in a distributed computing environment [30]. The information collected is correlated in real time to detect and analyze suspicious activity and processed in a centralized database. However, these systems have some weaknesses, as exemplified by G. Karantzas and C. Patsakis [31], for example, the comprehension of the artificial intelligence of the model created or the huge data ingestion required for accurate decision process.

2.3. Obfuscation Techniques

Years ago, a small modification in a piece of malware was able to circumvent antivirus software, so for each virus, we had several versions as if they were different developments (only manual analysis was able to find a common origin to assign a suitable virus name). However, the evolution of antivirus software with heuristics and AI, has been very effective in detecting variants of existing malware. The remaining challenge is to detect malware that uses obfuscation techniques, which is the focus of this research.

To avoid detection, several techniques have been developed. There are diverse techniques currently available to any attacker that can be applied individually or in combination to evade the defenses of any organization.

Self-encryption: The use of encryption algorithms or functions that contain a key included in the body of the malware that allows it to perform the encryption or decryption function in an automated manner [32].

Polymorphism: It uses a polymorphic engine to mutate itself and keep its original code intact [33,34,35].

Metamorphism: It can transform itself according to its capabilities to translate, edit, and rewrite its own code each time it infects a computer system [36,37].

Armouring: It is programmed to prevent any attempt at analysis or disassembly, making it impossible to know the original code [38].

Stealth: It completely or partially hides its presence in the system by intercepting requests of the operating system to interact with infected objects (boot sectors, file system elements, memory, etc.) [39,40].

Covert channels: It uses unauthorized communication channels and manipulates them in an unconventional way to transmit information undetected by anyone other than the entities operating the covert channel [41,42,43].

These obfuscation techniques make detection process very hard because the antivirus software is not able to read and analyze the code as in the case of any other static file. Moreover, due to the internal randomness of the obfuscation process, it is not possible to create signatures of the encrypted code because each sample of the same malware is different. In this paper, we propose to detect such kind of hidden malware by analyzing its behavior based on the chain of actions executed. Very recently evasion techniques have been applied to ransomware as well to avoid being detected while running. As an example, LockFile [44] performs file encryption by mapping the file to memory to avoid interaction with the hard drive during the process.

2.4. Threat Categorization

All adversaries will always execute a chain of actions [45,46,47] that are listed in the Cyber Kill Chain [48,49] (see Figure 1). This scheme allows to classify attacks into different levels of risk and to determine the perspective of the attacker at each moment. In addition, with the purpose of describing and categorizing adversary capabilities to improve system security, a structured matrix was created with the techniques and tactics used. The matrix known as MITRE ATT&CK (ATT&CK is a pseudonym of ATTACK) includes Tactics, Techniques, and Common Adversary Knowledge [50] and centralizes this information, allowing organizations to prevent possible threats to their computer systems.

The unification of these two schemes provides a real understanding of the type of technique and tactics used by an adversary at each stage of the attack process. By analyzing this information, we can extrapolate models that allow us to propose defensive strategies for detecting the actions of adversaries, and thus mitigate their consequences. Detection and classification systems (antivirus) on the market are very limited when performing these tasks in real time on malware that implement obfuscation techniques, because they mostly rely on static file analysis.

For these reasons, we have considered the implementation of a system based on the real-time analysis of the events [51,52,53] generated in the system and the classification of these events using the MITRE ATT&CK matrix and the Cyber Kill-Chain [54]. In this way, we were able to classify each event in the system, collecting all the events that have a malicious traceability.

3. Methodology and Implementation

Several Cybersecurity companies dedicate their efforts to the analysis of threats, developing and improving their detection and classification systems. These tasks resulting in the creation of a unique signature, the analysis of its operation at system and network level, or the study of its source code in depth. As a result, it is possible to obtain an updated knowledge database of known threats to defend computer systems. To evade these detection systems, adversaries can apply a multitude of obfuscation techniques, behavior modification and research of unknown techniques. Hence, antivirus companies need to create new signatures for each variant.

3.1. Related Work

This paper proposes a detection and threat classification system based on events generated by the operating system in real time, classifying them in the MITRE ATT&CK matrix of techniques and tactics together with their corresponding mapping in the Cyber Kill Chain. The criteria applied is based on the analysis of the events generated by different attacks, checking the tactic and technique where each of them is located within the MITRE ATT&CK matrix and therefore the phase within the Cyber Kill-Chain. Although there are existing works that have carried out research on event-based threat detection and classification systems [55,56], the one proposed is very effective regardless of the obfuscation method employed.

It is important to remark that this system does not pretend to replace the current antivirus systems, since they do their job effectively in most of the known cases of malware. However, it can be considered an extension to improve antivirus in the case of certain malicious code that is obfuscated and executed in memory. In most antivirus software, the analysis of code is only triggered by read or write actions on the hard drive.

The number of events collected during the execution of suspicious programs can be very large. Therefore, the current proposal is to match those events to the widely recognized MITRE ATT&CK matrix, and then classify as malicious any vector of attack containing techniques and actions considered of high-risk (from Execution/Exploitation). In future work, it will be possible to provide intelligence to the platform to automatically detect and classify malicious codes more accurately, for example considering the progression of events.

Table 1. Classification method using MITRE ATT&CK matrix and Cyber Kill Chain.

MITRE ATT&CK Tactics	Cyber Kill Chain	Number of High-Risk Techniques	Classified as Malicious
Reconnaissance	Reconnaissance	0/10	✕
Resource Development	Weaponization	0/7	✕
Initial Access	Delivery	0/9	✕
Execution	Exploitation	5/10	✓
Persistence	Installation	3/9	✓
Privilege Escalation	Actions on Objectives	4/10	✓
Defense Evasion	Installation	10/27	✓
Credential Access	Actions on Objectives	3/7	✓
Discovery	Reconnaissance	0/13	✕
Lateral Movement	Actions on Objectives	5/9	✓
Collection	Actions on Objectives	0/5	✕
Command and Control	Command and Control	8/13	✓
Exfiltration	Actions on Objectives	0/8	✕
Impact	Actions on Objectives	1/1	✓

shows the list of the 14 MITRE tactics, their correlation with the Cyber Kill Chain, the total number of techniques within each Tactic, and how many of those techniques are considered high-risk. It can be observed that for some Tactics, neither technique is considered of high-risk. For example, the Tactic Discovery has 13 different techniques, but none of them is considered malicious. On the other hand, in the case of Command and Control there 8 techniques out of 13 in this category, that are considered high-risk. Consequently, events associated with Discovery tactic will not be classified as malicious, while those associated with Command and Control will very likely classified as malicious (only 5 techniques will not). The reason for having white-listed techniques is because some techniques, such as command execution, can be used in a legitimate way. For example, system commands can be executed by any user and do not represent any threat.

The event-based detection system was implemented by combining of a set of open-source tools to automate the analysis of malicious techniques explained above. As a central core we used Sysmon [57,58], which runs as a resident service on the system and provides logging activity through the Windows event log. The communication between systems for sending information uses Winlogbeat [59] which is an open-source tool for sending Windows event logs to Kafka, which is an open-source distributed event streaming platform. For centralized information reception and analysis, a system based on ELK (ElasticSearch, Logstah and Kibana) was also implemented [60]. The first module (Logstah) is a server-side data processing pipeline that ingests multiple sources simultaneously, the second module (ElasticSearch) provides a data search and analytics engine and sends them to the third (Kibana) that facilitates the visualization of the data. Finally, all system information is consumed by Grafiki, which allows the creation of event graphs generated during the execution of malicious techniques in the system. We have named this detection system EDBS (Event Based Detection System). The block diagram of the system is shown on Figure 2.

The paper published by J. N. Praneeth and M. Sreedevi [61] similarly applied some of the proposed tools, but with very limited analysis of the results obtained. Only an unspecified piece of software is analyzed against VirusTotal using its hash code. The problem previously discussed can be observed in a similar way in other works which implement similar technologies, as the thesis published by M. Rasool [62] which uses VirusTotal as one of the central components in the classification of malware. In the research published by F. A. Bin Hamid Ali and Yee Yong Len [63], the authors describe a methodology for event-driven attack analysis based on signatures whereby their classification method only allows them to detect previously known attacks. Other publications, such as the thesis of U. Jain [64], rely on the analysis of events generated from the system of the type “Lateral Movement” but does not describe the traceability of the generated events and they are not assigned a risk level. In addition, there are other publications that implement possible architectures for threat detection and analysis [65,66,67]. These architectures are focused on the detection of suspicious activities in the network, and do not conceive the use of techniques that involve the execution of code directly in the memory of the victim system. Finally, there are papers that support the contextualized event-based analysis model, such as the one published by P. Giura and W. Wang [68] that describes both a methodology and a framework that can be implemented to perform the detection of different advanced attacks.

3.2. Implementation and Execution of Experimental Tests

The execution of the tests was performed based on known malicious code and techniques, which commercial antivirus systems already include in their signature databases or detect by means of heuristic algorithms that they have implemented. All of them were executed on a completely updated Microsoft Windows 10 operating system. The description of the malicious codes follows:

Code A: Powercat.ps1 is a PowerShell script equivalent to the Netcat tool that allows to open ports, to perform remote connections or do port scanning [69].

Code B: ConPtyShell.ps1 is a PowerShell script that uses the CreatePseudoConsole() function within Windows 10 version 1809 and allows the execution of a remote command console [70].

Code C: Invoke-PowershellTCP.ps1 is a PowerShell script that is part of the Nishang collection and allows the execution of a remote command console [71].

Code D: MeterpreterTCP.ps1 is a PowerShell version of one of the payloads generated by the Metasploit framework [72], that has a very wide set of functionalities and executes in memory, hence offering significant undetectability.

Code E: Mimikatz.ps1 is an open-source application that allows attackers to obtain information on the target system, such as stored credentials or Kerberos tickets [73].

Code F: Meterpreter.exe is the executable version of malicious code previously mentioned in “Code D”.

In addition, we made use of macros 4.0 (XML) in Excel documents, which are compatible with current Microsoft Office systems [74] and represent one of the techniques currently used to reduce the detection. We also added the use of signed operating system binaries, known as LOLBins [75], to conduct code execution once the document has been opened.

Code G: The Microsoft Excel document contains a macro 4.0 that runs a local command console using the MSBuild.exe binary.

Code H: The Microsoft Excel document contains a macro 4.0 that first downloads the file with malicious code using CertUtil.exe and then executes it using MSBuild.exe, as presented in the previous case.

Similarly, we have used Visual Basic for Applications (VBA) in Microsoft Office documents. Although these are very well-known, this has allowed us to obtain an approach for the results that we have obtained with our implementation.

Code I: The Microsoft Word document contains a Visual Basic for Applications (VBA) macro that download and execute a malicious code using PowerShell directly into memory.

In order to study the current state of the art in obfuscated malware detection, 11 commercial threat detection and classification (antivirus) systems were used. Different virtual machines were created with a clean updated Windows 10 installation and one antivirus at a time.

After basic initial testing of the aforementioned malware code, three different obfuscation methods were implemented to perform additional tests, consisting of encryption using AES algorithms, substitution of function names and known text strings, and encryption using XOR algorithm.

AES (Advanced Encryption Standard) encryption: It is a symmetric block cipher, i.e., it operates on groups of bits of fixed length and applies invariant transformations to them. The size of the blocks it handles is 128-bits and uses 128, 192, and 256-bit encryption keys.

String substitution: A simple way to evade antivirus systems in some cases is the substitution of known function names with random ones. As a result, it is possible to avoid identification by modifying the code signature that is compared with the one stored in the antivirus system’s database.

XOR encryption: It is a symmetric encryption based on the XOR logical operator and performs this operation bit by bit.

A general approach of the proposed methodology for data collection and analysis can be understood following the graph of events shown in Figure 3. This example corresponds to the execution of a Microsoft Word document that has embedded a macro (VBA) [76] with malicious code, that is Code I in the set of tests.

The graph of events shown in Figure 3, includes all the important events triggered by the execution of Code I. Therefore, the execution process can be described as follows:

• The user receives compressed file and proceeds to download it to the system.
• Launches WinRAR application to obtain the contents of the file.
• The result of the decompression is the file “test2.doc” (“prueba2.doc” in the graph) which contains the macro with malicious code embedded in it.
• It is executed by the user and the system calls the WINWORD.exe program to open the document.
• When the user enables the execution of the macros an error occurs in the WINWORD.exe binary that calls DW20.exe, which is responsible for compiling an error report.
• While the previous step occurs, the macro execution occurs which launches a command console on the system.

The mapping of the steps discussed above onto MITRE ATT&CK, and the Cyber Kill Chain are the following:

• The step 1 corresponds to the MITRE ATT&CK tactic “Initial Access” and technique “T1566-Phishing” and “Delivery” action in the Cyber Kill Chain.
• The step 2 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1204-User Execution” and “Exploitation” action in the Cyber Kill Chain.
• The step 3 corresponds to the MITRE ATT&CK tactic “Persistence” and technique “T1137-Office Application Startup” and “Installation” action in the Cyber Kill Chain.
• The step 4 corresponds to the MITRE ATT&CK tactic “Defense Evasion” and technique “T1055-Process Injection” and “Installation” action in the Cyber Kill Chain.
• The steps 5 and 6 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1059-Command-Line Interface” and “Installation” action in the Cyber Kill Chain.

In this example, the system detects several events that are matched to MITRE ATT&CK and Cyber Kill Chain, but most of them are low or medium risk. Technique “T1204” in steps 2 and 3 is part of the “Execution” tactic, but it is not considered high-risk, because it means that the user launched the execution of a program such as Word. Similarly, in step 4 the system captures a “Defense Evasion” tactic because of technique “T1055” which is neither a high-risk technique because it involves the execution of Dynamic Linked Libraries (DLLs), like almost all windows programs. Conversely, the latest event, also corresponding to the “Execution” tactic, is the one that actually triggers the alarm. Technique “T1059” is defined as type high-risk, because it means that the macro is launching a Command Tool or Power Shell for executing any kind of commands.

One of the features of Microsoft Word, is the ability to create multiple letters automatically out of a list of names and addresses stored in a database. This feature is called Mail Merge and can be used in conjunction with Open Database Connectivity (ODBC) to connect Word with a database server. It could be very useful to include a macro in that type of documents to verify, using a ping command, if there is connectivity with the database server before attempting to generate mail letters. Upon opening such document, the system will not trigger any alarm, because it contains a benign macro. The sequence of event will be very similar than before, but instead of technique “T1059” at the latest step, the system will be getting “T1018” (associated with ping command), which is a low-risk Discovery tactic that does not trigger the alarm.

Note that the procedure described with the help of the “Code I” can generalize to other types of attacks that execute in memory for avoiding traditional antivirus detection techniques. The methodology described, may potentially detect any attack defined by MITRE and will not reveal any risk in the case of benign events.

4. Experimental Results

The proposed methodology was evaluated for different categories of malware and obfuscation methods. The objective of the experiments is to show the effectiveness of event collection, the ability to match those events to the MITRE ATT&CK matrix and use that standard to determine if a threat is present of not. There are different attack vectors implemented in our experimental analysis that combine different tactics and techniques from the MITRE ATT&CK matrix. The evaluation is not based on individual malware samples, but on different attack vectors that ultimately implement actions that trigger system events detected by the proposed architecture. This is a more generalized assessment approach that targets non-existing malware or attack techniques and does not attempt to detect existing malware, because current antivirus can cope with such attacks.

Because of the obfuscation applied, standard antivirus programs are in disadvantage for detecting the malware tested. However, the event-based strategy can detect high-risk events, according to the MITRE specification, for the malware. Any attack that triggers high-risk system events is potentially detectable with the proposed approach regardless of the obfuscation method implemented by the attacker.

The event-bases detection system helps to perform complex analyses, such as experiments using Excel documents with macros 4.0, and signed operating system binaries. Code G and Code H include malware in the form of Excel macros. Their detail description follows:

Excel-Macro4.0-MSBuild-Cmd (Code G): The execution graph obtained after running Code G on the virtual machine is shown in Figure 4

Corresponding to the image in Figure 4, a detailed analysis of the events that have occurred in the system can be carried out, which allows for a comprehensive evaluation of the execution process:

• The user receives an Excel document which contains the macros 4.0 that allows malicious execution on the system.
• Opening the document, the system calls the EXCEL.exe program.
• Document “testapt1.xls” opens, and the user enables the execution of macros embedded in the document.
• Excel calls the binary MSBuild.exe that executes the commands in the system.
• As a result, command prompt is opened on the system.

The mapping of the steps described above onto MITRE ATT&CK and the Cyber Kill Chain is the following:

• The step 1 corresponds to the MITRE ATT&CK tactic “Initial Access” and technique “T1566-Phishing” and “Delivery” action in the Cyber Kill Chain.
• The step 2 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1204-User Execution” and “Exploitation” action in the Cyber Kill Chain.
• The step 3 corresponds to the MITRE ATT&CK tactic “Persistence” and technique “T1137-Office Application Startup” and “Installation” action in the Cyber Kill Chain.
• The step 4 corresponds to the MITRE ATT&CK tactic “Defense Evasion” and technique “T1127-Trusted Developer Utilities Proxy Execution” and “Installation” action in the Cyber Kill Chain.
• The step 5 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1059-Command-Line Interface” and “Exploitation” action in the Cyber Kill Chain.

Excel-Macro4.0-CertUtil-MSBuild-RevShell (Code H): The execution graph obtained after running this test is detailed in Figure 5:

The following points detail the execution process, which allows for a detailed analysis of the events that have occurred in the system.

• The user receives an Excel document containing macros 4.0 which allow malicious execution on the system.
• When the user proceeds to open the document, the system calls the EXCEL.exe program.
• The document “testapt2.xls” is opened and the user enables the execution of macros.
• The binary CertUtil.exe is executed at the beginning, which downloads the file with the malicious commands to the system.
• The file “readme2.txt” containing the malicious code is stored in a path on the computer.
• Afterwards, the binary MSBuild.exe is launched and it runs the contents of the file, in this case it contains C# code that executes PowerShell commands.
• It is possible to observe how “microsoft.powershell.commands.management” is executed.
• Policy compliance tests are carried out and also detected as events (8a, 8b), although they are not classified as risk events.

The mapping of the steps described above onto MITRE ATT&CK and the Cyber Kill Chain is the following:

• The step 1 corresponds to the MITRE ATT&CK tactic “Initial Access” and technique “T1566-Phishing” and “Delivery” action in the Cyber Kill Chain.
• The step 2 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1204-User Execution” and “Exploitation” action in the Cyber Kill Chain.
• The step 3 corresponds to the MITRE ATT&CK tactic “Persistence” and technique “T1137-Office Application Startup” and “Installation” action in the Cyber Kill Chain.
• The steps 4 and 5 correspond to the MITRE ATT&CK tactic “Defense Evasion” and technique “T1292-Indirect Command Execution” and “Installation” action in the Cyber Kill Chain.
• The step 6 corresponds to the MITRE ATT&CK tactic “Defense Evasion” and technique “T1127-Trusted Developer Utilities Proxy Execution” and “Installation” action in the Cyber Kill Chain.
• The step 7 corresponds to the MITRE ATT&CK tactic “Execution” and technique “T1059-Command and Scripting Interpreter” and “Exploitation” action in the Cyber Kill Chain.

The Event-based detection system (EBDS) proposed is able detect all malicious activity, because the malware always triggers an event classified as high-risk by MITRE. Even in the case of using obfuscation methods, EBDS can detect the attack once the original code is restored and key components (such as PowerShell) are executed. On the other hand, static analysis of the malicious file is not able to detect the key components because the payload is encrypted.

The comparison between the performance of different antivirus software, using techniques such as obfuscation and extracting malicious payloads in-memory, is shown in the following tables.

Table 2. Execution of malicious code without any encryption method.

AV/Test	A	B	C	D	E	F	G	H	I
PandaDome 20.01.00	✕	✕	✕	✕	✓	✓	✕	✕	✓
BitDefender 1.0.17.221	✕	✕	✓	✓	✓	✓	✕	✓	✓
TrendMicro 16.0	✕	✕	✕	✕	✓	✓	✓	✓	✓
Sophos 10.8.10.810	✕	✕	✕	✕	✕	✓	✕	✕	✓
Karspersky 21.2.16.590	✓	✕	✓	✓	✓	✓	✓	✓	✓
Avast 21.1.2443	✕	✕	✕	✕	✓	✓	✕	✕	✓
AVG 20.10	✕	✕	✕	✕	✓	✓	✕	✕	✓
Avira 15.0.2101.2070	✕	✕	✕	✓	✕	✕	✕	✕	✓
Eset 14.0.22.0	✕	✕	✓	✓	✓	✓	✕	✓	✓
BullGuard 21.0.387	✕	✕	✕	✓	✓	✓	✕	✕	✓
EBDS	✓	✓	✓	✓	✓	✓	✓	✓	✓

, without obfuscation, shows that most malware codes are detected by several commercial antivirus packages. However, after applying different obfuscation methods, detection is much harder (

Table 3. Execution of malicious code modified using AES encryption.

AV/Test	A	B	C	D	E	F	H
PandaDome 20.01.00	✕	✕	✕	✕	✕	✓	✕
BitDefender 1.0.17.221	✓	✓	✓	✓	✓	✓	✓
TrendMicro 16.0	✕	✕	✕	✕	✓	✓	✓
Sophos 10.8.10.810	✕	✕	✕	✕	✕	✓	✕
Karspersky 21.2.16.590	✓	✓	✓	✓	✓	✓	✓
Avast 21.1.2443	✕	✕	✕	✕	✓	✓	✕
AVG 20.10	✕	✕	✕	✕	✓	✓	✕
Avira 15.0.2101.2070	✕	✕	✕	✓	✕	✕	✕
Eset 14.0.22.0	✕	✕	✕	✓	✕	✕	✕
BullGuard 21.0.387	✕	✕	✓	✓	✓	✓	✓
EBDS	✓	✓	✓	✓	✓	✓	✓

,

Table 4. Execution of malicious code modified by applying string substitution.

AV/Test	A	B	C	D	E	H
PandaDome 20.01.00	✕	✕	✕	✕	✕	✕
BitDefender 1.0.17.221	✕	✕	✕	✓	✓	✕
TrendMicro 16.0	✕	✕	✕	✓	✓	✕
Sophos 10.8.10.810	✕	✕	✕	✕	✕	✕
Karspersky 21.2.16.590	✓	✕	✕	✓	✓	✓
Avast 21.1.2443	✕	✕	✕	✕	✕	✕
AVG 20.10	✕	✕	✕	✕	✕	✕
Avira 15.0.2101.2070	✕	✕	✕	✓	✕	✕
Eset 14.0.22.0	✕	✕	✕	✓	✓	✓
BullGuard 21.0.387	✕	✕	✕	✓	✕	✕
EBDS	✓	✓	✓	✓	✓	✓

and

Table 5. Execution of malicious code modified using XOR encryption.

AV/Test	A	B	C	D	H
PandaDome 20.01.00	✕	✕	✕	✕	✕
BitDefender 1.0.17.221	✕	✕	✓	✓	✓
TrendMicro 16.0	✕	✕	✕	✕	✕
Sophos 10.8.10.810	✕	✕	✕	✕	✕
Karspersky 21.2.16.590	✓	✕	✓	✓	✓
Avast 21.1.2443	✕	✕	✕	✕	✕
AVG 20.10	✕	✕	✕	✕	✕
Avira 15.0.2101.2070	✕	✕	✕	✓	✕
Eset 14.0.22.0	✕	✕	✓	✓	✓
BullGuard 21.0.387	✕	✕	✓	✓	✓
EBDS	✓	✓	✓	✓	✓

).

It is also important to mention that most antivirus are giving an alarm because they detect the obfuscation technique, giving a warning of “suspicious file”, but not detecting the malware itself. As an example, code B is not detected by BitDefender without obfuscation (Table 1), but it is detected when using AES encryption (Table 2), clearly because the use of encryption tools is raising the flag, since the payload cannot be read until executed. This detection strategy typically yields false positives if the user works with legitimate encrypted or coded files. We were able to prove such behavior by creating a harmless PowerShell code to just display a “hello world” message and obfuscating the file using AES; this file was also detected as malicious by several antivirus programs. The file was uploaded to VirusTotal [77,78] on 18 May 2021, and it was labeled as malicious by 9 antivirus programs.

5. Discussion

Traditional systems for detection and classification of threats in networks and computer systems are increasingly encountering difficulties for early detection of malicious code. This is because detection systems based on signature and heuristics have become outdated in the battle against novel evasion techniques. Therefore, this paper highlights the importance of implementing alternative detection methods, such as those based on the analysis and classification of events generated within the user’s computer. Through this type of analysis, it is possible to detect advanced techniques and establish security checkpoints for events generated in the system that may indicate that the system is being compromised.

The results presented in this paper correspond to commercial antivirus software and the proposed Event Management System EBDS, all running locally in the end-user computer. In large companies, it is possible to use SIEM (Security Information and Event Management) systems or corporate versions of antivirus software that centralize events, therefore attaining better results than a stand-alone antivirus. However, the work presented here was carried out with the focus on home, home-office, or small businesses, which need to rely on local computer detection.

The results show that, by applying obfuscation techniques, it is possible to hide malicious code from any detection system based on code signatures. Any encrypted code (malicious or benign) uploaded to VirusTotal will yield “safe” results, or maybe a warning related to fact that the file is encrypted. In addition, most advanced attacks manage to extract the obfuscated malicious code directly in the memory, without additional hard drive access, therefore avoiding a possible signature-based detection, because antivirus softwares scan files only while reading or writing to the hard drive.

Obfuscation techniques have no limits, there are multiple techniques and combination of techniques that can be applied. Since these techniques use a randomly generated key, every sample of obfuscated malware is different, and the creation of a general file signature is not possible. In addition, completely new malware, even without obfuscation, cannot be detected through signatures.

6. Conclusions

We have presented a detection system based on open-source tools, which is able to capture security critical events of the system. The proposed event-based analysis method, linked to MITRE Cyber Kill Chain, can stop the attack at the right moment, when reaching the state of “Execution/Exploitation”, preventing that PowerShell or Cmd are called.

Although standard antivirus programs are very effective and computationally efficient while detecting malware in general, they struggle with different obfuscation techniques. When obfuscation is applied to the malware, the proposed event-based strategy offers a second layer of analysis to improve detection rates compared to the set of non-detections and false positives of traditional antivirus systems alone.

For the purpose of experimentally validating the methodology proposed in this article, a comparative study has been carried out against 10 traditional detection systems (antivirus), always using the same set of samples and with the same obfuscation method in each case. Antivirus software was tested in different Windows 10 virtual machines, allowing the software to use all the potential of heuristics and real-time detection modules; nevertheless, it did not perform well for samples with obfuscation. On the other hand, the proposed methodology was able to detect the detect the sequence of events that were escalating in the severity.

As future work, Artificial Intelligence algorithms will be added to detect hazardous sequences of events, potentially being able to detect threats even in earlier stages of the attack. The current detection method was implemented by finding the state of “Execution/Exploitation”, but without exploiting details about sequences of states.