Memory and Fileless Malware Analysis

Introduction to Memory And Fileless Malware Analysis

Memory forensics is a method used by cybersecurity professionals to investigate suspicious activities inside a computer’s memory. Computer memory is also called Random Access Memory or RAM. It temporarily stores the information that programs need while they are running.

When you open a web browser, play a video, or run a program, your computer uses RAM to perform these tasks.

Some malware works mainly in memory instead of saving all its files on the hard drive. This makes it difficult for traditional antivirus software to detect the attack by scanning files alone.

Memory forensics helps security professionals find suspicious programs, hidden code, unusual activities, and signs of malware inside a computer’s memory.

Why Memory Forensics Is Important

Memory forensics helps investigators understand what happened on a computer during a suspected attack.

It can help them identify suspicious processes, examine malicious code, discover unusual network connections, and investigate malware that may not leave a normal file behind.

Example

Imagine that an employee receives an email containing a fake invoice. The employee opens the attachment, and the computer starts communicating with an unknown server.

A normal file scan may not find an obvious malicious program. However, a memory investigation may reveal a suspicious process or malicious code running inside another application.

This information helps the security team understand how the attack happened and what actions are needed to protect the computer.

Process Memory Fundamentals

A process is a program that is currently running on a computer. For example, when you open Google Chrome, Windows creates a process to run the browser.

Every process uses memory to store instructions, data, and other information needed to perform its tasks.

Understanding how process memory works helps cybersecurity professionals identify unusual activities.

Code Memory

Code memory contains the instructions that a computer’s processor executes.

For example, when you open Calculator in Windows, its instructions are loaded into memory so the application can perform calculations.

Malware may also place its instructions in memory. If these instructions appear in an unexpected location, an investigator may examine them to determine whether they are malicious.

Data Memory

Data memory stores information that a program needs while running.

For example, a browser may temporarily store information about open tabs, website content, and other working data.

Malware may use memory to store hidden commands, encrypted information, or details about a remote server.

Stack and Heap Memory

The stack stores information used during function calls, while the heap provides space for programs to store data when needed.

Both are normal parts of computer memory.

For example, a photo editing application may use heap memory to store image data while the user edits a picture.

Cybersecurity professionals examine these memory areas when investigating suspicious programs. However, using stack or heap memory is not itself a sign of malware.

Analyzing Running Processes

Process analysis means examining the programs currently running on a computer.

Security professionals check which processes are active, where they started, which programs launched them, and whether they perform unusual activities.

This helps investigators find programs that may be involved in a cyberattack.

Understanding Parent and Child Processes

A parent process is a program that starts another program. The new program is called a child process.

For example, when you open a command prompt manually, Windows starts a command shell. That shell may then start another program.

This relationship is normal in many situations. However, attackers sometimes use it to hide malicious activities.

Identifying Suspicious Processes

A process may deserve investigation if it has an unusual name, runs from an unexpected folder, starts other suspicious programs, or communicates with an unknown server.

For example, suppose a PDF reader suddenly starts PowerShell after opening a document received from an unknown sender.

This behavior may indicate that the document is attempting to run an unwanted command.

The investigator can examine the process history, command line, and related network activity to understand what happened.

Tools Used for Process Analysis

Security professionals commonly use Process Explorer and Process Monitor to examine running programs and their activities.

Memory analysis tools can provide additional information about the code and data stored inside these processes.

It is important to remember that an unfamiliar process is not always malicious. Investigators should check several pieces of evidence before reaching a conclusion.

Detecting Malicious Memory Regions

A memory region is a section of memory used by a program.

Some regions store data, while others contain instructions that the processor can execute.

Malware may hide its code in unexpected memory regions to make detection more difficult.

Understanding Memory Permissions

Memory permissions control how a program can use a memory region.

A region may allow reading, writing, or executing instructions.

For example, a writable region allows a program to change its contents. An executable region allows the processor to run instructions stored there.

A memory region that allows both writing and execution may deserve closer attention because it can be used to modify and run code.

However, some legitimate applications also use these permissions.

Finding Unusual Memory Regions

Investigators look for memory regions that do not match the normal behavior of the application.

For example, a web browser may contain an executable memory region that cannot be linked to an expected program module.

If the browser also shows unusual network activity, the investigator may examine that region for signs of injected code.

Why Further Investigation Is Necessary

An unusual memory region does not automatically mean that malware is present.

Some legitimate programs use special memory techniques to improve performance or run certain functions.

Investigators should compare memory permissions, code contents, process behavior, and other available evidence before deciding whether the region is suspicious.

Process Injection Concepts

Process injection is a technique in which code is placed or executed inside another running process.

Attackers may use this technique to hide their activities or make malicious code appear to be part of a trusted application.

This can make malware more difficult to identify through a simple list of running programs.

Why Attackers Use Process Injection

Attackers may try to hide malicious code inside a browser, office application, or another legitimate process.

For example, malware may attempt to run inside a browser process instead of appearing as a separate suspicious program.

A basic process list might show only the browser. However, memory analysis may reveal unusual executable code or other signs that the process has been modified.

How Security Professionals Detect Process Injection

Investigators look for unusual executable memory, unexpected code, suspicious thread activity, and memory regions that do not match expected program modules.

They also examine process relationships and system logs to understand how the suspicious activity started.

For example, if a familiar application contains unexpected executable code and begins communicating with an unknown server, the investigator may examine whether process injection occurred.

Several pieces of evidence should support the conclusion because legitimate software can also use complex memory techniques.

DLL Injection Analysis

DLL stands for Dynamic Link Library. A DLL is a file that contains code and functions that Windows programs can use.

Applications commonly load DLLs to perform tasks such as displaying windows, accessing system functions, and communicating with other components.

DLL injection happens when a DLL is introduced into another process through an unexpected or unauthorized method.

Understanding Normal DLL Loading

Many Windows applications use DLLs during normal operation.

For example, a photo editing program may load several libraries to display its interface, process images, and save files.

These libraries help applications share functions instead of including all the code in one file.

How Attackers May Abuse DLLs

An attacker may try to load a malicious DLL into a trusted application.

This allows the malicious code to run within the target process and may make the activity harder to notice.

For example, suppose a program loads an unfamiliar DLL from a temporary folder. If that DLL is also linked to suspicious network activity, a security professional may investigate it.

How to Investigate Suspicious DLLs

Investigators can check the DLL’s file location, digital signature, hash, and contents.

They can also compare the loaded libraries with those normally used by the application.

An unsigned DLL is not automatically malicious. Some legitimate software uses unsigned libraries.

The investigator must examine the complete situation before deciding whether the DLL is dangerous.

Reflective Loading Fundamentals

Reflective loading is a technique that allows a program module, such as a DLL, to be loaded into memory using a special method rather than the usual operating system loading process.

Some legitimate applications use custom loading techniques. Attackers may also use reflective loading to run malicious code without leaving a normal DLL file on the hard drive.

How Reflective Loading Works

In a simplified example, a program contains the contents of a DLL in memory.

A special loading routine prepares that content so it can work as a program module. The code may then run without following the usual file loading process.

Because of this, investigators may find code in memory without finding a matching DLL file in the expected location.

Example

Imagine that a running application contains code that looks like a DLL, but the investigator cannot find a matching library in the application’s normal module list.

This may indicate reflective loading or another special method of loading code into memory.

The investigator can examine the memory contents and related process activity to determine what happened.

Detecting Reflective Loading

Security professionals look for executable memory regions that contain code but are not linked to expected modules.

They may also examine thread activity, memory permissions, and unusual application behavior.

These clues can help identify suspicious activity, but further analysis is necessary because some legitimate applications use custom loading methods.

Fileless Malware Behavior

Fileless malware is malware that uses memory, scripts, or existing system tools to carry out malicious activities without relying entirely on a traditional executable file stored on the hard drive.

Some fileless attacks still use files during the infection process. The term describes attacks in which important malicious activities may happen mainly in memory or through legitimate system components.

How Fileless Malware Works

An attacker may trick a user into opening a malicious document or visiting a harmful website.

The attack may then use a legitimate system tool to run commands, download information, or communicate with an external server.

Because some of the activity happens in memory, a normal file scan may not reveal the complete attack.

Example

An employee receives an email claiming to contain an important company report.

After opening the attachment, a script runs through a legitimate Windows component. The script then attempts to contact an unknown server and obtain further instructions.

The computer may not contain an obvious malicious executable file. However, its process activity and memory contents may reveal signs of an attack.

Common Signs of Fileless Malware

Security professionals may investigate the following activities.

Unexpected use of PowerShell or other scripting tools.

Suspicious commands launched by office applications.

Unusual executable code inside a running process.

Unexpected connections to unknown internet servers.

Commands that appear heavily encoded or difficult to understand.

These activities do not always mean malware is present. Administrators and legitimate applications may use similar techniques. Investigators must check the context and supporting evidence.

PowerShell Based Malware Analysis

PowerShell is a Windows tool used to run commands and automate computer management tasks.

System administrators use it to manage users, configure systems, collect information, and perform repetitive tasks.

Attackers may also misuse PowerShell to run malicious commands, download harmful content, or perform activities directly in memory.

Why Attackers Use PowerShell

PowerShell is available on many Windows computers and can interact with different parts of the operating system.

Attackers may abuse these features to carry out malicious activities without creating a traditional executable file for every stage of an attack.

Example

Imagine that an employee opens a suspicious document.

The document launches PowerShell with a long command that is difficult to understand. PowerShell then attempts to connect to an unfamiliar internet server.

A security analyst would investigate which application started PowerShell, examine the command, and review any related network activity.

If a memory dump is available, the analyst may also examine the process for suspicious code or other evidence.

Signs of Suspicious PowerShell Activity

Security professionals may look for unexpected PowerShell processes, unusual command lines, suspicious downloads, and commands launched by applications that do not normally require scripting.

They may also review Windows event logs and endpoint security alerts.

Encoded commands are not always malicious. Some administrators use encoding for legitimate purposes.

The investigator should examine the command, its source, and its behavior before deciding whether an attack occurred.

Memory Dump Collection

A memory dump is a copy of some or all of a computer’s memory captured at a particular time.

Security professionals collect memory dumps to preserve information about running processes, loaded code, and other evidence that may disappear when the computer shuts down or a process stops.

Why Memory Dump Collection Is Important

Some malware operates mainly in memory. If the suspicious process stops running, important evidence may disappear.

For example, a company discovers that one of its computers is communicating with an unknown server.

The security team may collect a memory dump before making major changes to the affected computer. This allows investigators to examine the system’s memory later.

How Memory Dumps Are Collected

Investigators use trusted memory acquisition tools to capture the required memory according to an approved procedure.

They record the computer’s details, the collection time, the tool used, and other relevant information.

The captured dump is then stored securely for analysis.

Protecting Collected Evidence

A memory dump may contain sensitive information, including passwords, private messages, and confidential business data.

For this reason, access should be restricted to authorized personnel.

Investigators should also calculate a cryptographic hash, such as SHA-256, when appropriate to help verify that the evidence has not changed.

Memory collection can affect the running system, so investigators should understand the limitations of their tools and follow proper forensic procedures.

Memory Analysis with Volatility

Volatility is an open source tool used by cybersecurity professionals to investigate computer memory dumps.

It can help identify running processes, examine memory information, and investigate suspicious activity captured in a memory image.

Volatility supports different operating systems and memory formats, depending on the version and available plugins.

Examining Running Processes

One of the first tasks is to examine the processes found in the memory dump.

Investigators can review process names, identifiers, and relationships to identify unusual activity.

For example, a memory image may show that a document reader started PowerShell shortly before another process began communicating with an unfamiliar server.

This information may help the investigator understand how the suspicious activity began.

Examining Suspicious Memory

Volatility can help investigators examine available memory structures and identify unusual regions or code associated with processes.

For example, an application may contain an executable memory region that does not appear to match an expected program module.

The investigator can use this finding as a starting point for further analysis.

Investigating Network Activity

Depending on the operating system and the evidence preserved in the dump, Volatility may help identify network connections or related information.

For example, a suspicious process may be linked to a connection that also appears in firewall logs.

By comparing these sources, investigators can better understand whether the process may have been communicating with malicious infrastructure.

Limitations of Volatility

Volatility does not automatically detect every type of malware.

Its results depend on the quality of the memory dump, operating system support, available plugins, and the evidence present at the time of collection.

Investigators should compare its findings with security alerts, system logs, and other forensic evidence.

Identifying In Memory Indicators of Compromise

An Indicator of Compromise, or IOC, is a sign that may suggest a computer or network has been affected by a cyberattack.

In memory analysis, IOCs may include suspicious processes, unusual executable memory, unexpected code, strange command lines, or evidence of suspicious network activity.

These indicators help investigators decide what to examine more closely.

Suspicious Process Relationships

An unusual process relationship may indicate that an application has been used to start an unwanted program.

For example, a document reader launches PowerShell after opening an unexpected attachment.

If PowerShell then connects to an unfamiliar server, the sequence may indicate a malicious attempt.

Unusual Executable Memory

An executable memory region that cannot be linked to an expected program module may deserve investigation.

For example, a browser contains unexpected executable code and begins performing unusual network activity.

The investigator may examine the memory region for signs of process injection or another method of loading code.

Unexpected DLLs and Code

An unfamiliar DLL or unusual code inside a process can provide another clue.

Investigators may examine the file’s location, hash, signature, and contents to determine whether it belongs to legitimate software.

If the DLL is no longer available as a normal file, memory analysis may still provide useful information about its contents.

Suspicious Network Activity

Memory evidence may help investigators identify connections associated with a suspicious process.

For example, a scripting process communicates with an unfamiliar server shortly after a user opens a suspicious attachment.

The investigator can compare the connection with firewall records, endpoint alerts, and other available evidence.

Evidence of Fileless Malware

A process may contain suspicious code even when there is no obvious matching executable file on the hard drive.

This can support an investigation into fileless malware or code running through process injection.

However, the absence of a file does not prove that malware exists. Investigators need additional evidence to confirm what happened.

Bringing the Evidence Together

The strongest investigation combines several findings instead of relying on one suspicious sign.

For example, an investigator discovers that a suspicious document started PowerShell, PowerShell contacted an unknown server, and a related process contained unusual executable memory.

Together, these findings may indicate a malicious attempt to run code and communicate with an external system.

The investigator should document the evidence, explain the findings, and recommend suitable actions to contain the threat and protect other computers.

Conclusion

Memory and fileless malware analysis helps cybersecurity professionals investigate threats that may be difficult to find through ordinary file scanning.

By understanding process memory, examining running programs, detecting unusual memory regions, and investigating techniques such as process injection and reflective loading, analysts can identify suspicious activity more effectively.

Memory dump collection and tools such as Volatility help preserve and examine important evidence. PowerShell analysis and the identification of indicators of compromise provide additional information about how an attack may have occurred.

The most important lesson is to examine evidence carefully. One unusual process or memory region does not automatically prove that a computer is infected. Combining memory findings with system logs, network records, and other security evidence helps investigators reach more reliable conclusions.

Practical malware analysis should always be performed using authorized samples in a properly isolated environment.