What Every Programmer Should Know About Memory
Ulrich Drepper
Red Hat, Inc.
[email protected]
November 21, 2007
2.3 Other Main Memory Users
Beside the CPUs there are other system components which can access the main memory. High-performance cards such as network and mass-storage controllers cannot afford to pipe all the data they need or provide through the CPU. Instead, they read or write the data directly from/to the main memory (Direct Memory Access, DMA). In Figure 2.1 we can see that the cards can talk through the South- and Northbridge directly with the memory. Other buses, like USB, also require FSB bandwidth—even though they do not use DMA—since the Southbridge is connected to the Northbridge through the FSB, too.
While DMA is certainly beneficial, it means that there is more competition for the FSB bandwidth. In times with high DMA traffic the CPU might stall more than usual while waiting for data from the main memory. There are ways around this given the right hardware. With an architecture as in Figure 2.3 one can make sure the computation uses memory on nodes which are not affected by DMA. It is also possible to attach a Southbridge to each node, equally distributing the load on the FSB of all the nodes. There are a myriad of possibilities. In Section 6 we will introduce techniques and programming interfaces which help achieving the improvements which are possible in software.
Finally it should be mentioned that some cheap systems have graphics systems without separate, dedicated video RAM. Those systems use parts of the main memory as video RAM. Since access to the video RAM is frequent (for a 1024×768 display with 16 bpp at 60Hz we are talking 94MB/s) and system memory, unlike RAM on graphics cards, does not have two ports this can substantially influence the systems performance and especially the latency. It is best to ignore such systems when performance is a priority. They are more trouble than they are worth. People buying those machines know they will not get the best performance.
除了CPU，其他的系統組件可以訪問內存。高性能的網卡和大容量控制器並不允許他們的數據從CPU流過。相反，它們使用DMA（Direct Memory Access）的方式來直接讀或者寫內存，不經過CPU。在圖2.1中，我們可以看到設備通過南北橋和內存溝通。其他的總線，類似USB，也會搶佔FSB帶寬。 即使他們沒有使用DMA，但是南橋和北橋也是通過FSB聯繫到一起的。
使用DMA是十分的有利的，但這意味着與FSB總線帶寬會有競爭。當有較多的DMA數據傳輸時，CPU會比以往有更高的延遲，它需要等待從內存來的數據。我們可以用一些正確的硬件來解決這個問題。在圖2.3的架構中，我們的CPU使用直連的本地內存是不會被DMA所影響的。我們也可以在每個CPU節點上附加上一個南橋，使得所有節點負載均衡FSB的壓力。會有很多的技術方法。在第6節，將介紹一些技術和編陳的接口在軟件層次來幫助實現改善。
最後，需要提一下一些廉價的系統，它們的圖形系統沒有單獨的，專門的圖形內存。這些系統使用部分的主內存作爲圖形內存。因爲訪問圖形內存是頻繁的（一個1024*768，16bpp，60Hz的顯示設置需要94MB/s的傳輸速度），並且系統內存不像圖形顯卡上的內存，沒有兩個接口，這將會影響系統的性能，尤其是延遲。當專注性能的時候，我們最好放棄這樣的系統。它們製造的問題遠遠高於它們的價值。人們買它們的時候需要知道它們的性能很差。