lemp10 doesn't boot without additional RAM stick

[jacobgkau]

When using only the integrated RAM stick, lemp10 doesn't boot (power LED turns green, but never gets to the System76 splash screen.)

When an additional stick is installed, the OS reports the full amount of RAM (including the integrated 8GB) as being available.

[crawfxrd]

lemp is the only model with onboard RAM, so I probably messed up converting struct spd_info to struct mem_spd.

Maybe:

diff --git a/src/mainboard/system76/lemp10/romstage.c b/src/mainboard/system76/lemp10/romstage.c
index ed528e8bd074..6a916200b44a 100644
--- a/src/mainboard/system76/lemp10/romstage.c
+++ b/src/mainboard/system76/lemp10/romstage.c
@@ -13,7 +13,6 @@ void mainboard_memory_init_params(FSPM_UPD *mupd)
                .topo = MEM_TOPO_MIXED,
                .cbfs_index = 0,
                .smbus = {
-                       [0] = { .addr_dimm[0] = 0x50, },
                        [1] = { .addr_dimm[0] = 0x52, },
                },
        };

[jacobgkau]

It took a few minutes to get the repo cloned, but here is the cbmem output booted with an additional RAM stick installed: cbmem.txt

[crawfxrd]

No memory dimm at address A0

Yeah, so that change might fix it (0xA0 = 0x50 * 2).

I would need logs from booting with only onboard to be sure though.

[jacobgkau]

I will build with that change and check if it fixes it.

[jacobgkau]

I'm still seeing the issue after removing that line and flashing.

system76@pop-os:~/firmware-open/coreboot$ git diff
diff --git a/src/mainboard/system76/lemp10/romstage.c b/src/mainboard/system76/lemp10/romstage.c
index ed528e8bd0..6a916200b4 100644
--- a/src/mainboard/system76/lemp10/romstage.c
+++ b/src/mainboard/system76/lemp10/romstage.c
@@ -13,7 +13,6 @@ void mainboard_memory_init_params(FSPM_UPD *mupd)
                .topo = MEM_TOPO_MIXED,
                .cbfs_index = 0,
                .smbus = {
-                       [0] = { .addr_dimm[0] = 0x50, },
                        [1] = { .addr_dimm[0] = 0x52, },
                },
        };
system76@pop-os:~/firmware-open/coreboot$

[crawfxrd]

Can you set up debugging to get the coreboot log without the DIMM installed?

[jacobgkau]

Can you set up debugging to get the coreboot log without the DIMM installed?

I think I've done this before, but I don't remember the entire process. This would require the CH341A connected, right? Which scripts should I run from the host to see the debug logs? And is there anything I need to do to the firmware to enable debugging?

[crawfxrd]

You will need a Mega 2560.

In the EC, enable PARALLEL_DEBUG.

diff --git a/src/board/system76/common/common.mk b/src/board/system76/common/common.mk
index 6a12de8f206a..d888da6e2837 100644
--- a/src/board/system76/common/common.mk
+++ b/src/board/system76/common/common.mk
@@ -10,7 +10,7 @@
 CFLAGS+=-DLEVEL=4
 
 # Uncomment to enable debug logging over keyboard parallel port
-#CFLAGS+=-DPARALLEL_DEBUG
+CFLAGS+=-DPARALLEL_DEBUG
 
 # Uncomment to enable I2C debug on 0x76
 #CFLAGS+=-DI2C_DEBUGGER=0x76

And enable logging the coreboot console through the EC.

diff --git a/models/lemp10/coreboot.config b/models/lemp10/coreboot.config
index 071f8ac79010..db069246226d 100644
--- a/models/lemp10/coreboot.config
+++ b/models/lemp10/coreboot.config
@@ -19,4 +19,4 @@ CONFIG_SMMSTORE_V2=y
 CONFIG_TPM_MEASURED_BOOT=y
 CONFIG_USE_OPTION_TABLE=y
 CONFIG_VALIDATE_INTEL_DESCRIPTOR=y
-#CONFIG_CONSOLE_SYSTEM76_EC=y
+CONFIG_CONSOLE_SYSTEM76_EC=y

Then build and flash both. ec/doc/debugging.md has the full steps to set up the Arduino.

[jacobgkau]

After I flashed firmware with those two modifications to enable parallel port and coreboot logging, plus the one modification from before, I got one very slow boot (before I had the Mega 2560 connected), then I started having trouble booting at all, even with the additional RAM stick. Here are the logs I'm getting right now:

lemp10-ec-coreboot-debug-log-with-ram-stick.txt
lemp10-ec-coreboot-debug-log-without-ram-stick.txt

[jacobgkau]

Wait, I got it to go a bit farther after powering off by holding the power button instead of just unplugging. lemp10-ec-coreboot-debug-log-without-ram-stick-2.txt

It ends with this:

[16:53:59] coreboot-2021-12-21_1dc7780-dirty Tue Dec 21 17:31:30 UTC 2021 romstage starting (log level: 7)...
[16:53:59] POST: 0x00
[16:53:59] pm1_sts: 0800 pm1_en: 0000 pm1_cnt: 00001c00
[16:53:59] gpe0_sts[0]: 00000000 gpe0_en[0]: 00000000
[16:53:59] gpe0_sts[1]: 00000000 gpe0_en[1]: 00000000
[16:53:59] gpe0_sts[2]: 00000000 gpe0_en[2]: 00000000
[16:53:59] gpe0_sts[3]: 00000000 gpe0_en[3]: 00082000
[16:53:59] TCO_STS:   0000 0000
[16:53:59] GEN_PMCON: d0015038 00002200
[16:53:59] GBLRST_CAUSE: 00000000 00000000
[16:53:59] HPR_CAUSE0: 00000000
[16:53:59] prev_sleep_state 5
[16:53:59] CBFS: Found 'fspm.bin' @0x50dc0 size 0x9f000 in mcache @0xfef43094
[16:53:59] FMAP: area COREBOOT found @ 650200 (10157568 bytes)
[16:54:00] TPM: Digest of FMAP: COREBOOT CBFS: fspm.bin to PCR 2 logged
[16:54:00] POST: 0x34
[16:54:00] FMAP: area RW_MRC_CACHE found @ 600000 (65536 bytes)
[16:54:00] SPD index = 0
[16:54:00] CBFS: Found 'spd.bin' @0x4a580 size 0x200 in mcache @0xfef42fc8
[16:54:00] FMAP: area COREBOOT found @ 650200 (10157568 bytes)
[16:54:00] TPM: Digest of FMAP: COREBOOT CBFS: spd.bin to PCR 2 logged
[16:54:00] SPD: module type is DDR4
[16:54:00] SPD: module part number is M471A1G44AB0-CWE    
[16:54:00] SPD: banks 8, ranks 1, rows 17, columns 10, density 16384 Mb
[16:54:00] SPD: device width 16 bits, bus width 64 bits
[16:54:00] SPD: module size is 8192 MB (per channel)
[16:54:00] No memory dimm at address A4
[16:54:00] Length of SPD does not match for mixed topology!

[crawfxrd]

Flash back to 2021-07-20_93c2809 and get the cbmem log without the DIMM installed.

I expect the same error happens, but the code now calls die() instead of printk().

[jacobgkau]

Here it is: cbmem.txt

coreboot-2021-07-20_93c2809 Tue Jul 20 22:11:03 UTC 2021 romstage starting (log level: 7)...
pm1_sts: 0000 pm1_en: 0000 pm1_cnt: 00001c00
gpe0_sts[0]: 00000000 gpe0_en[0]: 00000000
gpe0_sts[1]: 00000000 gpe0_en[1]: 00000000
gpe0_sts[2]: 00000000 gpe0_en[2]: 00000000
gpe0_sts[3]: 00000000 gpe0_en[3]: 00000000
TCO_STS:   0000 0000
GEN_PMCON: d0015038 00002200
GBLRST_CAUSE: 00000000 00000000
HPR_CAUSE0: 00000000
prev_sleep_state 5
CBFS: Found 'fspm.bin' @0x4adc0 size 0x9f000 in mcache @0xfef42824
POST: 0x34
FMAP: area RW_MRC_CACHE found @ 600000 (65536 bytes)
SPD INDEX = 0
CBFS: Found 'spd.bin' @0x45000 size 0x200 in mcache @0xfef4279c
SPD: module type is DDR4
SPD: module part number is M471A1G44AB0-CWE    
SPD: banks 8, ranks 1, rows 17, columns 10, density 16384 Mb
SPD: device width 16 bits, bus width 64 bits
SPD: module size is 8192 MB (per channel)
No memory dimm at address A4
Mixed topology has incorrect length: 512 != 256.
POST: 0x36
POST: 0x92
POST: 0x98
CBMEM:
IMD: root @ 0x76fff000 254 entries.
IMD: root @ 0x76ffec00 62 entries.
FMAP: area RW_MRC_CACHE found @ 600000 (65536 bytes)
MRC: Checking cached data update for 'RW_MRC_CACHE'.
SF: Detected 00 0000 with sector size 0x1000, total 0x1000000
MRC: cache data 'RW_MRC_CACHE' needs update.
REGF update can't fit. Will empty.
MRC: updated 'RW_MRC_CACHE'.
1 DIMMs found
SMM Memory Map
SMRAM       : 0x7b800000 0x800000
 Subregion 0: 0x7b800000 0x200000
 Subregion 1: 0x7ba00000 0x200000
 Subregion 2: 0x7bc00000 0x400000
top_of_ram = 0x77000000
MTRR Range: Start=76000000 End=77000000 (Size 1000000)
MTRR Range: Start=7b800000 End=7c000000 (Size 800000)
MTRR Range: Start=ff000000 End=0 (Size 1000000)
CBFS: Found 'fallback/postcar' @0x12c980 size 0x48a8 in mcache @0xfef42904
Decompressing stage fallback/postcar @ 0x76c21fc0 (35200 bytes)
Loading module at 0x76c22000 with entry 0x76c22031. filesize: 0x45d0 memsize: 0x8940
Processing 159 relocs. Offset value of 0x74c22000
BS: romstage times (exec / console): total (unknown) / 1 ms

[crawfxrd]

Changing it back to an error log should fix boot:

diff --git a/src/soc/intel/common/block/memory/meminit.c b/src/soc/intel/common/block/memory/meminit.c
index 50c8b40df284..7f86f926d4c2 100644
--- a/src/soc/intel/common/block/memory/meminit.c
+++ b/src/soc/intel/common/block/memory/meminit.c
@@ -164,7 +164,8 @@ void mem_populate_channel_data(const struct soc_mem_cfg *soc_mem_cfg,
                 * topology.
                 */
                if (spd_md_len != spd_dimm_len)
-                       die("Length of SPD does not match for mixed topology!\n");
+                       printk(BIOS_ERR, "Mixed topology has incorrect length: %d != %d\n",
+                              spd_md_len, spd_dimm_len);
 
                data->spd_len = spd_md_len;
        }