One of the most common questions I get regarding HxD is about obtaining source code to understand and reproduce some of its basic functionality. The most frequent points of interest are how the RAM editor reads / writes memory of other running programs (i.e., processes) and how the disk editor reads from / writes to storage media.
Yesterday, when Mohamed A. Mansour, who will teach an Operating Systems class, suggested this information might be useful for his students, I thought it was a good opportunity to start my blog and write it up in a proper way.
This post is about reading memory that belongs to another running program (aka. process), and will be the first in a series of posts about the implementation and design of HxD.
Memory management basics
So, how do you read memory from another program? Let us first begin by introducing some common memory “layout” and access models. If you know already about the flat memory model, protected memory, and virtual memory you can skip to the next section.
Flat memory model
While memory systems can be complex, it is rare to encounter anything but a flat memory model, which essentially means that all main memory accessible to your program can be represented by a big one-dimensional array of bytes. To address a unit of memory (i.e., a byte) you simply index into the array.
In the illustrative table below, we assume a flat 32-bit memory model where each byte (uint8_t) out of theoretically 4 GiB can be addressed individually.
So each address (= array index) must be between 0 and 232-1 (= 4294967295). Represented in hexadecimal numbers, the lowest address is 0x0 and the highest address is 0xFFFFFFFF. We call this addressable range of memory the address space. Obviously, the actually accessible memory depends on the amount of installed physical memory (RAM).
Simple data types
If all memory can be thought of as being a big array, how do simple data types work?
In programming languages, variables are merely names for addresses that – with some “syntactic sugar” – ease access to memory. So if you want to store the height of a person in a byte-sized variable, you could do the following in C:
uint8_t person_height = 182; // 182 cm
Which is effectively just a shorthand for something like this:
main_memory[address_of_person_height] = 182; // 182 cm
You may also want to store a 32-bit wide integer that holds a color in the RGBA color space. For example: 33% red (0x55), 66% green (0xAA), 100% blue (0xFF), 0% transparency/alpha (0x00) could be expressed as hexadecimal number 0x55AAFF00.
uint32_t balloon_color = 0x55AAFF00;
In this case memory access would be slightly more complex. You would have to decompose the 32-bit value into 4 bytes, before you can write the value to main memory. Depending on the CPU the order of the bytes differs (see Endianness). In little endian systems such as x86-32 and x86-64 the order starts with the least significant end first.
0x55AAFF00 in little endian order would be this 4 byte sequence:
You can see that the hexadecimal notation lends itself well to split numbers into bytes, which is one reason why it is commonly used in hex editors but also in programming.
To write the 32-bit color value to main memory, each of those 4 bytes is written individually:
main_memory[address_of_balloon_color] = 0x00;
main_memory[address_of_balloon_color + 1] = 0xFF;
main_memory[address_of_balloon_color + 2] = 0xAA;
main_memory[address_of_balloon_color + 3] = 0x55;
In practice CPUs optimize access for commonly used byte multiples such as 4 byte (=32-bit) or 8 byte (=64-bit) data types, but conceptually memory access works as outlined above.
Arrays of non byte-sized elements
An important observation to make here is that addresses are always byte-array indices, even if the array element size is a multiple of bytes, such as for elements of type uint16_t (2 bytes) or double (8 bytes).
For example, assuming the byte array x and the unsigned 16-bit integer array y point to the same block of memory (and therefore their start addresses and lengths in bytes are the same):
uint8_t x; // byte array with 10 elements
uint16_t y; // uint16_t array with 5 elements
// we assume x and y point to the same memory block:
addr_of(x) == addr_of(y) // start addresses are the same
byte_length(x) == byte_length(y) == 10 // same byte lengths, though element_count(y) = 5!
Then, it holds that:
- addr_of(x) == addr_of(y)
- addr_of(x) == addr_of(y)
- addr_of(x) == addr_of(y)
- addr_of(x) == addr_of(y)
The memory layout of array x and y is illustrated in the following table, where each element is represented by a cell. Notice how elements of y cover two cells of x.
|array x (element size = 1 byte)||x||x||x||x||x||x||x||x||…|
|array y (element size = 2 bytes)||y||y||y||y||…|
In general, to get an address of an array-element a[i], you apply this formula:
address_of_a_i = addr_of(a) + i * array_element_size;
For example, the address of y would be:
address_of_y_3 = addr_of(y) + 3 * 2; // i = 3 and array_element_size = 2 (since y is a uint16_t array)
Hints for writing real code
The pseudo code above is meant to exemplify and explain what goes on behind the scenes. In production code you would use a function such as sizeof() in C to get the array_element_size. Also, in C to get an address you use the operator & instead of addr_of(), which is a pseudo code function used for clarity.
Depending on how your compiler performs data alignment, array elements may not be packed tightly together, but padding bytes might be inserted to improve alignment for performance reasons. In this case you have to account for the padding bytes that are usually appended after each element and increase array_element_size accordingly. So be sure to check the alignments and paddings your compiler uses and if sizeof() includes those additional bytes or not. This applies also to languages such as Delphi, not just C.
More safe and simple is to use normal array indexing, such as this in C: &a which gets you the address of the 6th element of the array a. Though, our point is to understand how this addressing works internally, so we are bound to be exposed to some technical details and caveats as mentioned above.
What about pointers?
A pointer is a variable of type integer only holding an index into the big one-dimensional memory array. When you dereference a pointer, you access the data through the array index (= memory address) stored in the pointer. The name pointer refers to the fact that pointer variables do not store the data itself but an index to it, i.e., they point to the data.
On computers that are programmed on the bare metal, such as microcontrollers (think of Arduino) or operating systems that do not protect memory (such as DOS or Win 3.1), all processes share the same address space. That means all running programs (aka. processes) have the same one-dimensional contiguous array available to them, as explained above.
This makes data exchange easy but can also cause erroneous or malicious overwriting of memory that belongs to another process – leading to stability and security problems. Protected memory introduces access barriers to avoid such issues.
To fully separate processes, address spaces are virtualized, such that it appears to each process as if there is a separate main memory per process, the virtual memory.
That is, each process has its own virtual memory array and can use the entire address range without colliding with another process. Therefore a virtual address is only meaningful inside a process and data cannot be shared (directly). The operating system takes care of mapping virtual addresses to actual physical addresses behind the scenes.
To reinforce the point consider this example: virtual address 0x1234 in process A points to the physical address 0x2000 while the same virtual address 0x1234 in process B points to the physical address 0x4000. So the virtual addresses 0x1234 in A and B do not collide despite having the same value because the operating system maps them to different physical addresses (⇒ different actual memory).
By being the middle man, the operating system can have fine grained control over memory and restrict access or grant it only to processes with special privileges. This makes foreign memory access a deliberate and restricted action and can improve security and stability.
Reading memory from another process under Windows
Given that modern operating systems use virtual protected memory, and therefore processes are totally isolated from each other, how can you access memory belonging to another process? The answer: application programming interfaces (API) provided by the operating system that are primarily designated for debugging purposes.
In the following I will present a small C++ program that dumps memory to the command line. For clarity I will only discuss the points directly relevant for accessing foreign memory and omit things such as string processing for pretty printing on the console, detailed error handling, or user interface (console based). You can download a fully working project for VS2010 (and up) that has all those features. Be sure to check its Readme.txt.
The following source code…
DWORD pid = /*set to desired process ID*/;
INT_PTR foreignProcessAddress = /*set to desired virtual address in process identified by pid*/;
const int bufferSize = 16;
SIZE_T bytesRead; // how many bytes could be read successfully
// we need at least PROCESS_VM_READ rights to the process to be allowed to read its memory
HANDLE processHandle = OpenProcess(PROCESS_VM_READ, false, pid);
if (processHandle != NULL)
if (ReadProcessMemory(processHandle, (LPCVOID)foreignProcessAddress, buffer, bufferSize, &bytesRead))
std::cout << bytes_to_hex_string(buffer, bytesRead) << std::endl;
std::cout << "Error reading process memory.\n";
std::cout << "Could not open process for memory reading.\n";
… would produce an output similar to this (actual output depends on the read data, here it is the start of a loaded executable file):
4D 5A 50 00 02 00 00 00 04 00 0F 00 FF FF 00 00
The basic principle underlying the source code is simple:
- You obtain the process ID from the process whose memory you want to access.
- You use OpenProcess to get a reading handle for the process.
- You use ReadProcessMemory to read memory starting at foreignProcessAddress and ending at foreignProcessAddress + bufferSize - 1.
- And finally you close the reading handle using CloseHandle.
The data is now in the byte-array buffer and can be printed to the screen as a string of hex-pairs using bytes_to_hex_string (see project download above for its implementation).
While the buffer is limited to 16 bytes, this is an arbitrary choice to simplify printing on the console. Usually you would read memory in larger chunks, for example using the memory page size (see GetSystemInfo).
What is most puzzling about this code is how to obtain a process ID or find out what memory address to read. Again the example project shows one way how this could be done.
Obtaining process IDs
The task manager can show the PID of each running process in a separate column. If this column is not yet visible, select the tab “processes”, then select “View|Choose columns…”, and tick the checkbox “PID (process-ID)”.
You can also list all running processes using Windows-API functions or get a process ID from a GUI window. I will present some of this in a future blog post.
Determining valid or useful memory addresses
If you know already the address at which the foreign process stores the data you are interested in, this is a no-brainer: just set foreignProcessAddress accordingly.
If you want to explore the memory like you can do in HxD, you need to enumerate all accessible memory regions (using VirtualQueryEx ), or simply read through all the virtual memory address space from 0 to end, and just output those ReadProcessMemory calls that do not return an error.
Similarly, you can scan for valid memory regions and then perform a text search on the buffer. This is also a candidate for some future blog post.
Other operating systems
As mentioned before, programs running under operating systems that do not protect memory or isolate processes through virtual memory, can just address the entire main memory and read it using pointers. Sometimes addressing can be complicated through a segmented memory model as found in DOS, but this is hardly relevant today.
“Safe” languages such as Java do not allow to specify random memory addresses — by design they only ever allow references to the owned memory — and therefore would still prohibit reading foreign memory even in a low policing OS. Given a special OS API (as presented above) or external library this can be circumvented, but not with native means of the Java language, e.g., arrays or references.
Finally, like most modern operating systems, Linux also uses a virtual protected memory model. However, my experience with reading foreign process memory under Linux is limited and I wouldn’t want to give an untested example.
Phew, this first post turned out much longer and formal than I intended it to be, but I thought that some background and clarity was necessary to really understand how to use the functions. I hope this post is useful to you and will give you an idea what’s going on in case of unexpected results.
Do you have an equally functioning example for Linux as the one I provided for Windows? Add a comment below and share your knowledge :).
Also, I welcome comments on my content presentation or writing, I am new to blogging and keen to learn.