Virtual Memory and IO - Part I

Virtual memory is a type of memory indirection that has allowed systems to enforce usage restrictions, security and gives a simplified memory view for programs. Of course, it comes with a cost of performance and restrictions on memory sharing. Let’s understand what virtual memory means for device IO i.e interaction with devices connected to CPU.

Mainly there are two types of IO memory operations -

  • Programmed IO where CPU cycles are used to transfer data from memory to device Programmed

  • Direct Memory Access based DMA

There could also be an IOMMU, which we’ll discuss in later parts. In programmed IO, there are two types: Memory Mapped IO (MMIO) and Port Mapped IO (PMIO).

Focusing on MMIO, these are for devices that have registers that can be mapped into the program’s virtual space i.e. these devices have a permanent physical address (most cases). The physical address range shows up in /proc/iomem in Linux and with the RAMMap tool in Windows. This is also the reason why in 32 bit systems RAM of 4GB cannot be completely mapped as some of the address space is taken by MMIOs (3GB Barrier). There are API’s that kernels provide to read/write from this memory once you have mapped them into your program. Since there is cache involved in memory accesses, these API’s add memory barriers appropriately to provide order guarantees.

To sum it up, to access a device’s register X using MMIO -

  • Read the datasheet of the system and device to figure out the physical address at which X is mapped
  • Map that physical address to your process virtual space
  • Perform R/W on the virtual address using special IO API’s
Written on March 7, 2020