Skip to content

Known issue: L2 cache flushing due to StarFive 7100 architecture #1

Closed
Closed
Known issue: L2 cache flushing due to StarFive 7100 architecture#1
Assignees
tekkamanninja
Labels
known issue

Description

@pdp7
pdp7
opened on Apr 11, 2021

NOTE: I am reposting my forum post here for greater visibility.

The BeagleV beta boards have the StarFive 7100 SoC which was from a limited multi-project wafer run. We only have enough chips to produce 300 BeagleV beta boards. StarFive should be releasing a public datasheet next week for the 7100. The 7100 has 2x StarFive U74 cores.

StarFive will be putting in a full production order an updated design called the 7110. This will have 4x SiFive U74 cores as well as Imagination GPU and PCIe controller.

This brings me to design issues in the 7100:

  • L2 cache controller has a bug where it incorrectly raises the L2 DirFail interrupt. The only work around for this is to mask the L2 cache interrupt on the PLIC.
  • All the peripherals in the 7100 are on a non-coherent bus. StarFive refers to this as the sysbus port. This requires the drivers to flush L2 cache in many instances where it is not necessary on other SoC’s. This design choice reduces the performance of the 7100.

StarFive tells me that they have resolved the L2 cache controller DirFail interrupt bug in the 7110. And more importantly they have improved the SoC architecture. High performance peripherals like the Gigabit Ethernet and USB3 controller will be connected to a cache-coherent bus (StarFive refers to it as the front port).

My understanding so far is that it may be difficult to upstream the 7100 support given that the peripherals are not coherent. I would very much like to get input from you and others in the beta as to what the correct way to implement the drivers might be.

I also need to work with StarFive to create some diagrams of the SoC architecture that shows exactly how the different IP blocks are interconnected and whether or not those connections are coherent.

Thank you,
Drew Fustini
BeagleBoard.org Foundation

Activity

  1. pdp7
    self-assigned this
    on Apr 11, 2021
  2. pdp7
    added
    7100 SoC
    known issue
    on Apr 11, 2021

  3. pdp7 commented on Apr 11, 2021

    @pdp7
    pdp7
    on Apr 11, 2021
    CollaboratorAuthor

    Please see this issue for discussion of L2 cache flushing specific to the u-boot code:

  4. pdp7 commented on Apr 19, 2021

    @pdp7
    pdp7
    on Apr 19, 2021
    CollaboratorAuthor

    refer to starfive-tech/sft-riscv-linux-5.10#2

  5. pdp7
    closed this as completedon Apr 19, 2021
  6. pdp7
    reopened this on Apr 21, 2021

  7. xypron commented on Apr 26, 2021

    @xypron
    xypron
    on Apr 26, 2021
    Collaborator

    Would the following observation match the problem you reported?

    # dd if=/dev/sda of=/dev/null bs=16M count=64

    leading to

    [ 1972.045204] L2CACHE: flush64 out of range: 2080200000(80000), skip flush
[ 1972.091354] L2CACHE: flush64 out of range: 2080200000(80000), skip flush
[ 1972.133556] L2CACHE: flush64 out of range: 2080200000(80000), skip flush
[ 1972.176994] L2CACHE: flush64 out of range: 2080200000(80000), skip flush
[ 1972.219621] L2CACHE: flush64 out of range: 2080200000(80000), skip flush
[ 1972.262798] L2CACHE: flush64 out of range: 2080200000(80000), skip flush

  8. xypron commented on Apr 26, 2021

    @xypron
    xypron
    on Apr 26, 2021
    Collaborator

    The problem seems to stop us from using an USB connected drive:

    [   24.670434] usbcore: registered new interface driver uas
[   24.670760] L2CACHE: flush64 out of range: 2080200000(24), skip flush
[   24.671320] L2CACHE: flush64 out of range: 2080200000(60), skip flush
[   24.671868] scsi 0:0:0:0: Direct-Access     Samsung  SSD 860 EVO 1TB  4101 PQ: 0 ANSI: 6
[   24.672010] L2CACHE: flush64 out of range: 2080200000(ff), skip flush
[   24.672478] L2CACHE: flush64 out of range: 2080200000(ff), skip flush
[   24.672909] L2CACHE: flush64 out of range: 2080200000(ff), skip flush
[   24.673345] L2CACHE: flush64 out of range: 2080200000(ff), skip flush
[   24.673877] L2CACHE: flush64 out of range: 2080200000(1000), skip flush
[   24.676117] L2CACHE: flush64 out of range: 2080200000(20), skip flush
[   24.676452] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   24.676961] L2CACHE: flush64 out of range: 2080200000(40), skip flush
[   24.677387] L2CACHE: flush64 out of range: 2080200000(40), skip flush
[   24.677768] L2CACHE: flush64 out of range: 2080200000(40), skip flush
[   24.678083] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   24.678101] sd 0:0:0:0: [sda] 4096-byte physical blocks
[   24.678200] L2CACHE: flush64 out of range: 2080200000(4), skip flush
[   24.688780] sd 0:0:0:0: [sda] Write Protect is off
[   24.688802] sd 0:0:0:0: [sda] Mode Sense: 53 00 00 08
[   24.688941] L2CACHE: flush64 out of range: 2080200000(4), skip flush
[   24.689404] L2CACHE: flush64 out of range: 2080200000(20), skip flush
[   24.689835] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   24.689953] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.690538] L2CACHE: flush64 out of range: 2080200000(40), skip flush
[   24.690897] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes)
[   24.760643] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.760941] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.761213] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.761484] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.761754] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.762066] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.762352] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.762690] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.763905] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.764259] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.764616] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.764972] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.765326] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.765689] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.766044] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.766419] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.767054] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.767402] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.767750] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.768100] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.768274] Unable to handle kernel paging request at virtual address 0000005f81a4e6b2
[   24.768291] Oops [#1]
[   24.768296] Modules linked in: uas usb_storage
[   24.768326] CPU: 0 PID: 176 Comm: systemd-udevd Tainted: G        W         5.10.6+ #26
[   24.768336] epc: ffffffdf81a4e6b2 ra : ffffffe00090bf54 sp : ffffffe084a6f980
[   24.768342]  gp : ffffffe0018416a8 tp : ffffffe080615040 t0 : 0000000000000040
[   24.768349]  t1 : ffffffe1fed7d000 t2 : 0000000000000000 s0 : ffffffe084a6f9b0
[   24.768360] L2CACHE: flush64 out of range: 2080200000(200), skip flush
[   24.768367]  s1 : ffffffe08603bc00 a0 : ffffffe08603bc00 a1 : ffffffe001843228
[   24.768374]  a2 : ffffffe001843228 a3 : 0000000000000000 a4 : ffffffe08019f800
[   24.768381]  a5 : ffffffdf81a4e6b2 a6 : 0000000000000000 a7 : 000000000000001f
[   24.768389]  s2 : ffffffe0807f6000 s3 : 0000000000000000 s4 : ffffffe084c3b680
[   24.768397]  s5 : ffffffe0018ab6d8 s6 : ffffffe00004b480 s7 : ffffffe0807f61ac
[   24.768405]  s8 : ffffffe00004b480 s9 : ffffffe00004b330 s10: ffffffe001844208
[   24.768410]  s11: ffffffe084a73318 t3 : 000000000000007f t4 : 0000000000000402
[   24.768417]  t5 : ffffffe0842f65a8 t6 : ffffffe0842f65b0
[   24.768423] status: 0000000200000120 badaddr: 0000005f81a4e6b2 cause: 000000000000000c
[   24.768551] ---[ end trace 6c3bac6aaa60c4fa ]---
[   24.768564] Kernel panic - not syncing: Fatal exception in interrupt
[   24.768574] SMP: stopping secondary CPUs
[   27.015675] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

  9. pdp7 commented on Apr 26, 2021

    @pdp7
    pdp7
    on Apr 26, 2021
    CollaboratorAuthor

    @xypron I believe this is related to starfive-tech/Fedora_on_StarFive#3

    @tekkamanninja @MichaelZhuxx would you be able to comment as to your current understanding of this issue?

  10. MichaelZhuxx commented on Apr 28, 2021

    @MichaelZhuxx
    MichaelZhuxx
    on Apr 28, 2021
    Collaborator

    yes,that's a know issue about L2 cache

  11. pdp7 commented on Apr 28, 2021

    @pdp7
    pdp7
    on Apr 28, 2021
    CollaboratorAuthor

    FYI - note on JH7100 Cache Coherence has been published
    https://github.com/starfive-tech/beaglev_doc/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf

  12. tekkamanninja
    added 3 commits that reference this issue on May 13, 2021

    ovl: fix leaked dentry

    d587cfa

    scsi: qla2xxx: Reserve extra IRQ vectors

    0f86d66

    irqchip/gic-v3: Do not enable irqs when handling spurious interrups

    3f72d37

  13. 320 remaining items

  14. esmil
    added 15 commits that reference this issue on Nov 19, 2021

    drm/msm: prevent NULL dereference in msm_gpu_crashstate_capture()

    660446f

    PM: hibernate: Get block device exclusively in swsusp_check()

    df0b686

    btrfs: do not take the uuid_mutex in btrfs_rm_device

    6039fcd

    netfilter: conntrack: set on IPS_ASSURED if flows enters internal str…

    6c7a147

    net/mlx5: Accept devlink user input after driver initialization complete

    9cc25e8

    scs: Release kasan vmalloc poison in scs_free process

    69dce45

    kernel/sched: Fix sched_fork() access an invalid sched_task_group

    3869eec

    blk-wbt: prevent NULL pointer dereference in wb_timer_fn

    3e6e11f

    mt76: mt7615: fix monitor mode tear down crash

    1f71b42

    mt76: connac: fix possible NULL pointer dereference in mt76_connac_ge…

    4393933

    blk-cgroup: synchronize blkg creation against policy deactivation

    af25dbc

    serial: imx: fix detach/attach of serial console

    977b112

    NFS: Fix an Oops in pnfs_mark_request_commit()

    d9bd973

    scsi: qla2xxx: Fix gnl list corruption

    411ca50

    scsi: ufs: core: Fix NULL pointer dereference

    e022fea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

known issue

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions