Reverse Engineering Malware: A Deep Dive

Introduction

Reverse engineering malware is a critical skill for security researchers, threat analysts, and incident responders. By dissecting malicious software, we can uncover its true capabilities, identify vulnerabilities, and develop more effective countermeasures. In this post, we will conduct a deep analysis of a real-world malware sample using Ghidra, an open-source reverse engineering tool widely used by professionals in the field.

Selecting the Malware Sample

For this analysis, we have chosen Agent Tesla, a well-known keylogger and Remote Access Trojan (RAT) that has been actively used in cyber espionage campaigns. Typically delivered through phishing emails, Agent Tesla has targeted businesses and individuals alike, often masquerading as legitimate software attachments to evade detection. Our goal is to reverse engineer this sample to understand its inner workings and identify potential indicators of compromise (IoCs).

Setting Up the Analysis Environment

Before analyzing the malware, it's essential to establish a controlled and secure environment to prevent accidental execution. This includes:

• A virtual machine (VM): Running the malware in an isolated environment using VMware or VirtualBox to prevent system compromise.

• A sandboxing tool: Utilizing platforms such as FLARE VM, Any.Run, or Cuckoo Sandbox to monitor behavior dynamically.

• Reverse engineering tools: Ghidra for static analysis, complemented by IDA Free, x64dbg for debugging, and CyberChef for decoding obfuscated data.

By ensuring a proper setup, we minimize risks while maximizing the depth of our analysis.

Loading the Malware into Ghidra

Once the environment is set, we begin by loading the Agent Tesla sample into Ghidra. After creating a new project and importing the binary, we allow Ghidra’s auto-analysis to process the file. This reveals valuable insights into function structures, symbol tables, and code relationships. In cases where the malware is packed, unpacking it first using a debugger like x64dbg is often necessary to retrieve the original executable.

Analyzing the Malware’s Behavior

String Analysis

Examining embedded strings is one of the fastest ways to gather intelligence on a malware sample. Within Ghidra, navigating to Window > Defined Strings presents potential clues such as:

• Hardcoded URLs and IP addresses for command-and-control (C2) servers.

• Registry modifications and file paths used for persistence.

• Base64-encoded or XOR-encrypted payloads that indicate obfuscation techniques.

In our analysis, we discovered that Agent Tesla stores credentials for email-based exfiltration within the binary, making it possible to extract C2 communication details directly from the strings section.

API Calls and Function Imports

Windows API calls are essential for malware execution, revealing the sample’s capabilities. By reviewing the Import Address Table (IAT), we can infer key functionalities. Some notable functions used by Agent Tesla include:

• GetAsyncKeyState: Capturing keystrokes to log user input.

• WriteProcessMemory: Injecting code into other processes for stealthy execution.

• InternetOpenUrlA: Communicating with remote servers for data exfiltration.

Identifying these API calls helps us understand how the malware interacts with the operating system and executes its malicious tasks.

Control Flow and Decompilation

To better grasp the malware’s internal logic, we analyze its control flow graph (CFG) and use Ghidra’s Decompiler to reconstruct higher-level pseudocode. This allows us to bypass obfuscation techniques and focus on the core functionality. Agent Tesla employs process injection to evade detection, frequently leveraging CreateRemoteThread to execute payloads in legitimate processes like explorer.exe.

Additionally, by tracing the execution flow, we identified functions responsible for credential theft from browsers and email clients. The malware retrieves stored passwords and transmits them via SMTP to an attacker-controlled email address.

Extracting Indicators of Compromise (IoCs)

Throughout the analysis, we document findings that can aid in detection and response efforts. Key IoCs extracted from our Agent Tesla sample include:

• C2 domains and IPs: Hardcoded within the binary and used for exfiltration.

• File paths: Locations where malware drops secondary payloads.

• Registry keys: Entries modified for persistence.

• Mutex values: Unique identifiers used to prevent multiple instances from running simultaneously.

These IoCs provide security teams with actionable intelligence for threat hunting and incident response.

Mitigation Strategies

To defend against Agent Tesla and similar malware, organizations should adopt a multi-layered security approach:

• Advanced email filtering: Deploying security gateways to block phishing attempts and suspicious attachments.

• Endpoint Detection and Response (EDR): Leveraging behavioral analysis to detect keylogging or unauthorized process injections.

• User awareness training: Educating employees about social engineering tactics used to deliver malware.

• Threat intelligence integration: Using IoCs from previous attacks to proactively block known threats.

Conclusion

Reverse engineering malware is an invaluable process that enables security professionals to stay ahead of evolving cyber threats. In this analysis, we used Ghidra to dissect Agent Tesla, uncovering its keylogging, credential theft, and data exfiltration mechanisms. By continuously studying emerging malware families, we strengthen our collective defense against cybercriminals.