文章/memory-latency-and-bandwidth-comparison

#灌水1 分钟阅读

内存性能大比较!

对比多套硬件的内存延迟、带宽和 NUMA 表现。

Intel® Memory Latency Checker v3.11b

AMD Ryzen 7 9700X

-> % sudo lshw -c memory -short
H/W path                  Device          Class          Description
====================================================================
/0/0                                      memory         64KiB BIOS
/0/d                                      memory         640KiB L1 cache
/0/e                                      memory         8MiB L2 cache
/0/f                                      memory         32MiB L3 cache
/0/13                                     memory         48GiB System Memory
/0/13/0                                   memory         [empty]
/0/13/1                                   memory         24GiB DIMM Synchronous Unbuffered (Unregistered) 5600 MHz (0.2 ns)
/0/13/2                                   memory         [empty]
/0/13/3                                   memory         24GiB DIMM Synchronous Unbuffered (Unregistered) 5600 MHz (0.2 ns)
-> % sudo ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0          80.3

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      58929.8
3:1 Reads-Writes :      66580.4
2:1 Reads-Writes :      66096.6
1:1 Reads-Writes :      55213.5
Stream-triad like:      67975.4

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        58812.5

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  178.12    58921.6
 00002  175.72    59148.9
 00008  173.88    58971.0
 00015  172.73    58965.9
 00050  185.50    59092.4
 00100  193.09    58822.9
 00200  185.97    58934.0
 00300  130.40    47925.9
 00400  111.70    37966.8
 00500  105.12    31714.0
 00700  102.12    23952.2
 01000  100.80    17469.9
 01300   99.64    13890.6
 01700   96.24    10925.5
 02500   94.55     7768.6
 03500   93.30     5820.1
 05000   93.43     4298.0
 09000   93.51     2705.6
 20000   92.08     1611.9

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        15.7
Local Socket L2->L2 HITM latency        15.8

AMD AI MAX 395+

-> % sudo lshw -c memory -short
H/W path          Device          Class          Description
============================================================
/0/0                              memory         64KiB BIOS
/0/b                              memory         1280KiB L1 cache
/0/c                              memory         16MiB L2 cache
/0/d                              memory         64MiB L3 cache
/0/11                             memory         128GiB System Memory
/0/11/0                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/1                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/2                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/3                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/4                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/5                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/6                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/7                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
-> % sudo ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         154.4

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      117081.5
3:1 Reads-Writes :      151323.4
2:1 Reads-Writes :      160109.6
1:1 Reads-Writes :      179796.4
Stream-triad like:      134033.9

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        117020.3

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  206.43   116980.0
 00002  205.88   116874.9
 00008  203.66   115473.1
 00015  204.44   115052.1
 00050  207.31   118860.5
 00100  203.07   115144.3
 00200  217.25    95711.7
 00300  196.43    89988.4
 00400  180.71    71684.1
 00500  173.23    59571.4
 00700  169.98    44459.9
 01000  163.40    32452.3
 01300  161.46    25534.4
 01700  160.37    19887.2
 02500  159.66    13889.4
 03500  159.96    10142.6
 05000  158.44     7281.4
 09000  160.18     4242.9
 20000  159.33     2144.0

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        17.5
Local Socket L2->L2 HITM latency        17.5

Intel Ultra 9 285H

-> % sudo lshw -c memory -short
H/W path          Device          Class          Description
============================================================
/0/0                              memory         64KiB BIOS
/0/12                             memory         64GiB System Memory
/0/12/0                           memory         32GiB SODIMM Synchronous 5600 MHz (0.2 ns)
/0/12/1                           memory         32GiB SODIMM Synchronous 5600 MHz (0.2 ns)
/0/1d                             memory         288KiB L1 cache
/0/1e                             memory         384KiB L1 cache
/0/1f                             memory         18MiB L2 cache
/0/20                             memory         24MiB L3 cache
/0/21                             memory         320KiB L1 cache
/0/22                             memory         640KiB L1 cache
/0/23                             memory         10MiB L2 cache
/0/24                             memory         24MiB L3 cache
/0/100/14.2                       memory         RAM memory
-> % sudo ./Linux/mlc
[sudo] password for user:
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for sequential access (in ns)...
                Numa node
Numa node            0
       0         270.1

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      78440.3
3:1 Reads-Writes :      76891.5
2:1 Reads-Writes :      77205.5
1:1 Reads-Writes :      77033.2
Stream-triad like:      77061.9

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        78578.3

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  251.56    76195.1
 00002  249.19    75876.3
 00008  288.68    80638.9
 00015  275.13    79408.2
 00050  246.77    75827.7
 00100  209.39    61399.6
 00200  193.59    46888.4
 00300  193.61    34877.7
 00400  186.20    27353.6
 00500  183.14    22485.5
 00700  181.26    17103.6
 01000  179.27    12473.8
 01300  178.00     9898.2
 01700  177.66     7782.8
 02500  177.11     5550.5
 03500  177.62     4089.2
 05000  177.50     3024.4
 09000  178.82     1829.0
 20000  180.87     1019.9

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        91.0
Local Socket L2->L2 HITM latency        92.1

AMD EPYC 7K62

-> % sudo lshw -c memory -short
[sudo] password for user:
H/W path              Device           Class          Description
=================================================================
/0/0                                   memory         64KiB BIOS
/0/21                                  memory         224GiB System Memory
/0/21/0                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/1                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/2                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/3                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/4                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/5                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/6                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/7                                memory         [empty]
/0/24                                  memory         3MiB L1 cache
/0/25                                  memory         24MiB L2 cache
/0/26                                  memory         192MiB L3 cache
-> % ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         110.7

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      17503.5
3:1 Reads-Writes :      16796.5
2:1 Reads-Writes :      16591.0
1:1 Reads-Writes :      16305.2
Stream-triad like:      16432.0

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        17127.6

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  501.66    17519.5
 00002  504.00    17527.5
 00008  532.26    17561.6
 00015  528.77    17552.4
 00050  538.20    17536.7
 00100  531.86    17565.1
 00200  532.29    17566.0
 00300  532.23    17560.9
 00400  532.54    17570.5
 00500  533.94    17542.5
 00700  526.42    17573.1
 01000  519.98    17563.5
 01300  498.84    17526.6
 01700  386.58    17527.4
 02500  174.14    15569.8
 03500  145.14    11328.5
 05000  135.52     8108.2
 09000  124.54     4806.6
 20000  119.60     2478.4

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        22.4
Local Socket L2->L2 HITM latency        22.7
[Sat Aug 30 11:21:54 2025] [Hardware Error]: event severity: fatal
[Sat Aug 30 11:21:54 2025] [Hardware Error]:  Error 0, type: fatal
[Sat Aug 30 11:21:54 2025] [Hardware Error]:  fru_text: DIMM Locate:  P0H0
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   section_type: memory error
[Sat Aug 30 11:21:54 2025] [Hardware Error]:    error_status: Storage error in DRAM memory (0x0000000000040400)
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   node:0 card:4 module:0
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   error_type: 15, physical memory map-out event

真奇怪,又 g 了,明明修好了才是

重启了一下:

[Sat Aug 30 14:17:19 2025] BERT: Error records from previous boot:
[Sat Aug 30 14:17:19 2025] [Hardware Error]: It has been corrected by h/w and requires no further action
[Sat Aug 30 14:17:19 2025] [Hardware Error]: event severity: corrected
[Sat Aug 30 14:17:19 2025] [Hardware Error]:  Error 0, type: corrected
[Sat Aug 30 14:17:19 2025] [Hardware Error]:  fru_text: DIMM Locate:  P0H0
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   section_type: memory error
[Sat Aug 30 14:17:19 2025] [Hardware Error]:    error_status: Storage error in DRAM memory (0x0000000000040400)
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   physical_address: 0x0000003f56549180
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   node:0 card:4 module:0 rank:0 bank:6 row:63538 column:528
[Sat Aug 30 14:17:19 2025] BERT: Total records found: 1
-> % free -mh
               total        used        free      shared  buff/cache   available
Mem:           251Gi       4.2Gi       248Gi       5.0Mi       645Mi       247Gi
Swap:          4.0Gi          0B       4.0Gi

居然自己好了。

-> % sudo mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         119.2

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      136823.2
3:1 Reads-Writes :      134545.9
2:1 Reads-Writes :      136703.3
1:1 Reads-Writes :      135919.7
Stream-triad like:      136468.7

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        136846.2

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  680.01   136402.3
 00002  680.09   136451.1
 00008  603.64   130970.7
 00015  380.77   105365.8
 00050  383.47   105430.4
 00100  390.13   105452.3
 00200  400.41   105823.8
 00300  216.26   107479.2
 00400  148.23    88973.3
 00500  139.61    73057.0
 00700  133.20    53527.2
 01000  124.36    38585.8
 01300  117.81    29940.8
 01700  116.19    23132.7
 02500  114.84    15983.0
 03500  114.62    11608.9
 05000  114.74     8304.9
 09000  115.69     4871.7
 20000  117.22     2491.8

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        22.4
Local Socket L2->L2 HITM latency        22.7

image.png

AMD Ryzen 7 9800X3D

16GB * 2, DDR5 6000MT/s

(0) > .\Windows\mlc.exe
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0          78.6

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      56582.8
3:1 Reads-Writes :      62128.6
2:1 Reads-Writes :      61323.4
1:1 Reads-Writes :      52797.4
Stream-triad like:      64569.7

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        55632.8

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  220.10    56594.2
 00002  248.74    56311.8
 00008  337.96    56143.3
 00015  286.18    55749.9
 00050  246.07    56395.8
 00100  219.09    56282.8
 00200  177.87    53891.6
 00300  113.36    41023.7
 00400  123.68    31897.4
 00500  117.87    25665.6
 00700  117.57    18838.9
 01000  110.93    13795.8
 01300  114.69    10782.7
 01700  106.25     8418.5
 02500  108.61     5935.5
 03500  134.22     4178.1
 05000  120.32     3221.0
 09000   99.62     2164.5
 20000  101.26     1314.9

Measuring cache-to-cache transfer latency (in ns)...
Unable to enable large page allocation
Using small pages for allocating buffers
Local Socket L2->L2 HIT  latency        19.1
Local Socket L2->L2 HITM latency        19.5

相关阅读

Memo关闭侧信道攻击缓解,make Linux great again!Memotime-warp-test.c: check TSC synchronity on x86 CPUsMemoArchLinux NVIDIA GPU 容器启动失败

讨论

邮箱用于身份识别和回复通知,并由 Waline 存储。