Lesson 26: Malware Analysis — Tools and Methodologies

What You Will Learn in This Lesson

• The difference between static and dynamic malware analysis and when to use each
• How to build a safe, isolated malware analysis lab from scratch
• Essential open-source and commercial tools for each analysis phase
• A step-by-step methodology for triaging a suspicious binary found on an endpoint

Why Malware Analysis Matters for Your Security Team

A suspicious executable lands on an employee workstation in your bank's treasury department. Your EDR flags it, quarantines it, and the alert hits the SOC dashboard. Now what? You can submit the hash to VirusTotal and wait for a vendor label, or you can crack it open yourself and answer the questions that actually matter: What data did it target? Did it establish persistence? Has it already exfiltrated anything? Malware analysis is the discipline that turns a mysterious binary into actionable intelligence.

There are two broad approaches. Static analysis examines the malware without executing it — you inspect strings, headers, imports, and disassembled code. Dynamic analysis runs the sample in a controlled environment and observes its behavior: network connections, file modifications, registry changes, and process creation. Most real investigations blend both. You start static to get initial indicators, pivot to dynamic to confirm behavior, then return to static to understand the deeper logic.

Setting Up a Safe Analysis Lab

Before you touch a live sample, you need an environment where detonation cannot escape. The cardinal rule: never analyze malware on a production machine or a network segment connected to corporate infrastructure. A proper lab uses virtualization with snapshots, host-only networking, and ideally a dedicated physical host that is air-gapped or behind a tightly controlled firewall.

Here is a practical lab architecture that works well for mid-size financial institutions:

# Recommended Lab Setup
─────────────────────────────────────────────
Host Machine (Linux or Windows)
├── VMware Workstation Pro / VirtualBox
│   ├── Analysis VM: Windows 10 (snapshot: "clean")
│   │   ├── Flare-VM toolkit (auto-installs 140+ tools)
│   │   └── Network: Host-Only adapter
│   ├── Analysis VM: REMnux (Linux-based malware toolkit)
│   │   └── Network: Host-Only adapter
│   └── INetSim VM (simulates internet services)
│       └── Network: Host-Only adapter, acts as DNS/HTTP/SMTP
─────────────────────────────────────────────
# All VMs on the same host-only network (e.g., 192.168.56.0/24)
# INetSim answers DNS/HTTP so malware "thinks" it has internet
# Snapshot before every detonation, revert after

Two purpose-built distributions dramatically speed up setup. Flare-VM from Mandiant transforms a Windows VM into a full analysis workstation — it installs debuggers, disassemblers, PE editors, network analyzers, and more via a single Chocolatey-based script. REMnux is its Linux counterpart, pre-loaded with tools for analyzing documents, scripts, shellcode, and network traffic. Together, they give you both a Windows detonation target and a Linux analysis powerhouse.

Static Analysis — Reading the Malware Without Running It

Static analysis is your first pass. It is safer, faster, and often reveals enough to classify the sample and extract indicators of compromise (IOCs). Start with these steps:

1. Hash and reputation check. Calculate MD5, SHA-1, and SHA-256 hashes and query threat intelligence platforms. This takes seconds and may immediately tell you the malware family.

# On REMnux or any Linux box
sha256sum suspicious.exe
# Output: a1b2c3d4e5f6... suspicious.exe

# Query VirusTotal via API
curl -s "https://www.virustotal.com/api/v3/files/a1b2c3d4e5f6..." \
  -H "x-apikey: YOUR_API_KEY" | jq '.data.attributes.last_analysis_stats'

2. File type and metadata. Verify the file is what it claims to be. Attackers routinely disguise executables as PDFs or images.

file suspicious.exe
# Output: PE32 executable (GUI) Intel 80386, for MS Windows

exiftool suspicious.exe
# Reveals compile timestamps, original filename, embedded resources

peframe suspicious.exe
# Extracts imports, exports, sections, suspicious API calls, URLs, IPs

3. String extraction. Embedded strings reveal C2 domains, file paths, registry keys, error messages, and sometimes hardcoded credentials.

strings -a suspicious.exe | grep -iE "(http|https|ftp|\.com|\.net|cmd\.exe|powershell)"
# Also try FLOSS (FLARE Obfuscated String Solver) for encoded strings
floss suspicious.exe

4. PE structure analysis. Examine the Portable Executable headers for anomalies: unusually high entropy in sections (suggests packing), suspicious imports (VirtualAlloc, CreateRemoteThread, WriteProcessMemory), and mismatched compile timestamps.

Practical example: During an incident at a Saudi payment processor, the SOC received an alert on a file named salary_update_Q1.pdf.exe. Static analysis showed it was a UPX-packed PE with strings referencing hxxps://update-payroll[.]xyz/gate.php and imports for InternetOpenA and CryptEncrypt. Before even running it, the team had a C2 domain to block, evidence of data exfiltration intent, and a packing method to reverse — all within 15 minutes.

Dynamic Analysis — Watching the Malware in Action

Once static analysis gives you a baseline, dynamic analysis reveals actual behavior. Revert your Windows VM to a clean snapshot, transfer the sample (use a password-protected ZIP), and prepare your monitoring tools before execution.

Essential monitoring stack:

• Process Monitor (ProcMon) — captures real-time file system, registry, and process/thread activity. Filter on your sample's process name to reduce noise.
• Process Hacker / Process Explorer — visualizes process trees, loaded DLLs, handles, and network connections per process.
• Wireshark — captures all network traffic on the host-only interface. INetSim on the adjacent VM answers requests so you see the full conversation.
• Regshot — takes before/after snapshots of the registry and filesystem, then diffs them to show exactly what changed.
• API Monitor — hooks Windows API calls in real-time, showing you the exact functions the malware invokes with their parameters.

# Dynamic analysis workflow
1. Revert VM to clean snapshot
2. Start ProcMon (filter: ProcessName = suspicious.exe)
3. Start Wireshark on host-only interface
4. Take Regshot "1st shot"
5. Execute the sample
6. Wait 5-10 minutes (some malware has delayed execution)
7. Take Regshot "2nd shot" → Compare
8. Stop captures, save ProcMon log (.PML) and Wireshark pcap
9. Analyze results
10. Revert VM to clean snapshot

For scaling dynamic analysis, automated sandboxes save enormous time. Cuckoo Sandbox (now evolved into CAPEv2) automates detonation, behavioral logging, network capture, and report generation. It can process dozens of samples per hour and integrates with YARA rules, MITRE ATT&CK mapping, and threat intelligence feeds. For teams that prefer a managed solution, Any.Run provides interactive cloud sandboxing where you can click through dialog boxes and observe behavior in real-time.

Building a Triage Methodology

Speed matters during incidents. Here is a repeatable five-stage triage process your team can adopt:

Stage 1 — Identification. Collect the sample, calculate hashes, and check threat intel. If the hash is already known, pull the existing report and jump to containment. Time: 5 minutes.

Stage 2 — Surface static analysis. Run file, peframe, strings, and FLOSS. Document file type, packing, embedded IOCs, and suspicious imports. Time: 15 minutes.

Stage 3 — Automated sandbox detonation. Submit to CAPEv2 or Any.Run. While it runs, continue static work. Review the sandbox report for network IOCs, dropped files, and ATT&CK techniques. Time: 10-20 minutes.

Stage 4 — Manual dynamic analysis. If the sandbox misses behavior (common with environment-aware malware), run manually with ProcMon, Wireshark, and Regshot. Check for anti-analysis techniques: VM detection, sleep timers, sandbox evasion. Time: 30-60 minutes.

Stage 5 — Deep static / reverse engineering. Use Ghidra or IDA Pro to disassemble and decompile critical functions. Map the malware's capabilities to MITRE ATT&CK. Write YARA rules for detection. Time: varies (hours to days for complex samples).

Not every sample needs all five stages. A known commodity trojan identified at Stage 1 can be handled immediately. A novel, targeted implant found in your SWIFT environment warrants full Stage 5 treatment.

Essential Tool Reference

Here is a quick-reference table of the tools every analyst should have ready:

┌─────────────────────┬──────────────────────────────┬────────────┐
│ Phase               │ Tool                         │ Cost       │
├─────────────────────┼──────────────────────────────┼────────────┤
│ Hash / Reputation   │ VirusTotal, MalwareBazaar    │ Free       │
│ File Identification │ file, exiftool, Detect-It-Ez │ Free       │
│ String Extraction   │ strings, FLOSS               │ Free       │
│ PE Analysis         │ peframe, pestudio, CFF Expl. │ Free       │
│ Disassembly         │ Ghidra                       │ Free (NSA) │
│ Disassembly         │ IDA Pro                      │ Commercial │
│ Debugging           │ x64dbg, WinDbg               │ Free       │
│ Sandbox             │ CAPEv2, Any.Run              │ Free/Paid  │
│ Network Capture     │ Wireshark, NetworkMiner       │ Free       │
│ Behavior Monitoring │ ProcMon, API Monitor         │ Free       │
│ Registry Diffing    │ Regshot                      │ Free       │
│ YARA Rules          │ yara, yarGen                 │ Free       │
│ Internet Simulation │ INetSim, FakeDNS             │ Free       │
└─────────────────────┴──────────────────────────────┴────────────┘

Connection to Saudi Regulatory Requirements

Both SAMA's Cyber Security Framework (CSCC) and NCA's Essential Cybersecurity Controls (ECC) require financial institutions to maintain incident response and threat analysis capabilities. SAMA CSCC Domain 3 (Cybersecurity Operations and Technology) explicitly mandates malware protection controls, and expects regulated entities to be able to investigate and characterize threats — not just block them at the perimeter. The NCA ECC control family for Threat Management requires organizations to implement mechanisms for detecting, analyzing, and responding to malicious code. Building an in-house or outsourced malware analysis capability directly satisfies these requirements and strengthens your posture during SAMA's periodic cybersecurity assessments. Additionally, the ability to extract IOCs from malware and share them with sector peers through platforms like the Saudi CERT (CERT-SA) fulfills the information sharing expectations outlined in both frameworks.

Common Mistakes to Avoid

• Analyzing on a connected machine. Even "just looking at strings" can trigger embedded scripts. Always use an isolated VM. One misconfigured network adapter can expose your corporate network to the sample's payload.
• Skipping snapshots before detonation. Without a clean snapshot, you cannot reliably diff what changed. Worse, your analysis VM becomes contaminated and future results are unreliable. Snapshot, detonate, analyze, revert — every single time.
• Relying solely on automated sandboxes. Modern malware checks for sandbox artifacts (VM tools, known MAC address prefixes, unrealistic system uptime, lack of user documents). If your only analysis is "submit to sandbox and read the report," you will miss evasive samples. Always be ready to fall back to manual dynamic analysis.

Lesson Summary

• Malware analysis combines static (examining code without execution) and dynamic (observing runtime behavior) techniques in a structured triage methodology
• A safe analysis lab requires isolated VMs, host-only networking, internet simulation (INetSim), and disciplined snapshot management
• A five-stage triage process — from hash lookup to deep reverse engineering — ensures you invest the right level of effort per sample and extract actionable IOCs quickly

Next Lesson

In the next lesson we will cover: Digital Forensics Fundamentals — how to acquire, preserve, and analyze digital evidence from disk images, memory dumps, and log files while maintaining a chain of custody that holds up under regulatory scrutiny.

Ready to apply these concepts in your organization? Contact Fyntralink for a complimentary SAMA Cyber Maturity Assessment.