build using -mcmodel = medany

[12101111]

I'm trying to port this crate to a new board targeting riscv64gc.

The original link script is https://github.com/kendryte/kendryte-standalone-sdk/blob/master/lds/kendryte.ld

My memory.x:

MEMORY
{
  RAM : ORIGIN = 0x80000000, LENGTH = (6 * 1024 * 1024)
}

This device don't have flash in its cpu address space. So I replace FLASH in link.x to RAM

I try to rebuild asm.S in riscv-rt and riscv

riscv64-unknown-elf-gcc -mabi=lp64 -march=rv64gc  -mcmodel=medany -c asm.S -o bin/riscv-rt.o
riscv64-unknown-elf-ar crs bin/riscv64gc-unknown-none-elf.a bin/riscv-rt.o

Build and output is

note: rust-lld: error: H:\board\K210\rust-test\target\riscv64gc-unknown-none-elf\debug\deps\libriscv_rt-5900706913d05900.rlib(riscv-rt.o):(.init+0x0): relocation R_RISCV_HI20 out of range: 2147487184 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\rust-test\target\riscv64gc-unknown-none-elf\debug\deps\libriscv_rt-5900706913d05900.rlib(riscv-rt.o):(.init+0x4): relocation R_RISCV_LO12_I out of range: 2147487184 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\rust-test\target\riscv64gc-unknown-none-elf\debug\deps\libriscv_rt-5900706913d05900.rlib(riscv-rt.o):(.init+0x8): relocation R_RISCV_HI20 out of range: 2153775104 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\rust-test\target\riscv64gc-unknown-none-elf\debug\deps\libriscv_rt-5900706913d05900.rlib(riscv-rt.o):(.init+0xC): relocation R_RISCV_LO12_I out of range: 2153775104 is not in [-2147483648, 2147483647]
          rust-lld: error: src\main.rs:5:(.text.main+0x4): relocation R_RISCV_HI20 out of range: 2147483900 is not in [-2147483648, 2147483647]
          rust-lld: error: src\main.rs:5:(.text.main+0x8): relocation R_RISCV_LO12_I out of range: 2147483900 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv\src\register\mtvec.rs:44:(.text._ZN5riscv8register5mtvec5write17hf4b8713aebc80d04E+0x30): relocation R_RISCV_HI20 out of range: 2147484944 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv\src\register\mtvec.rs:44:(.text._ZN5riscv8register5mtvec5write17hf4b8713aebc80d04E+0x34): relocation R_RISCV_LO12_I out of range: 2147484944 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:224:(.init.rust+0x4): relocation R_RISCV_HI20 out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:224:(.init.rust+0x8): relocation R_RISCV_LO12_I out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:224:(.init.rust+0xC): relocation R_RISCV_HI20 out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:224:(.init.rust+0x10): relocation R_RISCV_LO12_I out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x1E): relocation R_RISCV_HI20 out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x22): relocation R_RISCV_LO12_I out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x26): relocation R_RISCV_HI20 out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x2A): relocation R_RISCV_LO12_I out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x2E): relocation R_RISCV_HI20 out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:225:(.init.rust+0x32): relocation R_RISCV_LO12_I out of range: 2147485136 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:232:(.init.rust+0x46): relocation R_RISCV_HI20 out of range: 2147483788 is not in [-2147483648, 2147483647]
          rust-lld: error: H:\board\K210\riscv-rt\src\lib.rs:232:(.init.rust+0x4A): relocation R_RISCV_LO12_I out of range: 2147483788 is not in [-2147483648, 2147483647]
          rust-lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

It seems rustc don't have a option to build using -mcmodel=medany and default to -mcmodel = medlow

In sifive's blog,-mcmodel=medlow generate R_RISCV_HI20/R_RISCV_LO12_I and -mcmodel=medany generate R_RISCV_PCREL_HI20/R_RISCV_PCREL_LO12_I

[Disasm]

Unfortunately, addresses >=0x80000000 are not supported at the moment. Some details here: rust-embedded/wg#218 (comment)
I saw some patches in LLVM upstream, but haven't tested them yet.

Nice to see someone putting effort in Kendryte! At the moment I'm working on similar task: support for HiFive Unleashed board, which is also RV64GC.

[Dantistnfs]

Probably you could link it with riscv64-unknown-elf-ld
see my cargo config: https://github.com/Dantistnfs/riscv_k210

[Disasm]

@Dantistnfs it may work for you because you don't use riscv-rt and any static constants like strings and arrays.

[Dantistnfs]

@Disasm
I think that's better than nothing, at least you can do small experiments, before patches arrive
And yes, when I tried to use arrays/strings I got "Relocation truncated to fit: R_RISCV_HI20"

[Disasm]

I figured out that on K210 it's possible to run code from address 0x40000000 instead of 0x80000000. In the future, RAM-only linking will be possible with region aliases.

I've started working on k210-pac. At the moment, the only peripherals supported are GPIOHS and UARTHS, as they are pretty the same as in HiFive1. I'm planning to recover peripheral information from the SDK source code. If someone wants to help me - PRs welcome :)

[vmedea]

I figured out that on K210 it's possible to run code from address 0x40000000 instead of 0x80000000

we found out that flw and fld from the 0x4xxxxxxxx range always return 0 ! this sabotages pretty much all floating point computation—surprisingly though, it looks like 0x80000000 is aliased at 0xffffffff80000000 so this memory.x works:

MEMORY
{
    SRAM : ORIGIN = 0xffffffff80000000, LENGTH = 6M
    AI_SRAM : ORIGIN = 0xffffffff80600000, LENGTH = 2M
    SRAM_NOCACHE : ORIGIN = 0x40000000, LENGTH = 6M
    AI_SRAM_NOCACHE : ORIGIN = 0x40600000, LENGTH = 2M
}

REGION_ALIAS("REGION_TEXT", SRAM);
REGION_ALIAS("REGION_RODATA", SRAM);
REGION_ALIAS("REGION_DATA", SRAM);
REGION_ALIAS("REGION_BSS", SRAM);
REGION_ALIAS("REGION_HEAP", SRAM);
REGION_ALIAS("REGION_STACK", SRAM);

[Disasm]

@medusacle Is this behavior reproducible in plain C and official SDK?

[vmedea]

yes, it seems so, the following main.c, compiled in the SDK context:

#include <stdio.h>
#include "sysctl.h"
#include "uarths.h"
#include "sleep.h"

static float fp_const_a = 1.2345f;

int main(void)
{
    if (current_coreid() == 0)
    {
        uarths_init();
        usleep(100000);
        float *ptr_a = &fp_const_a;
        printf("%f at %p\r\n", *ptr_a, ptr_a);
        float *alt_a = (float*)(((uintptr_t)&fp_const_a) - 0x40000000);
        printf("%f at %p\r\n", *alt_a, alt_a);
    }
    while (1)
        asm volatile ("wfi");
    return 0;
}

(loading the resulting binary at 0x40000000 to skip the cache and assure it's not simply a cache coherency issue)
prints:

1.234500 at 0x8000e628
0.000000 at 0x4000e628

(for doubles I was not able to reproduce it here because the SDK is compiled with -march=rv64imafc so they go through the soft path, which like integers and character constants works fine)

[Disasm]

Thanks for investigating the issue! It's a serious argument against using 0x40000000 base address. If 0xffffffff80000000 base works fine, let's use it as a workaround.

[Disasm]

Updated relevant crates to reflect these changes.

[Disasm]

@medusacle Thanks again! This hack with 0xffffffff80000000 address is really great!

[WRansohoff]

Using 0xffffffff80000000 for the RAM addresses also seems to work for me, but I think it might cause some issues debugging with GDB and OpenOCD. When the program is actually executed in GDB, the leading ffffffff address bytes are often fully or partially dropped, which prevents GDB from finding the debugging symbols.

For example:

$ riscv64-unknown-elf-nm target/riscv64gc-unknown-none-elf/debug/test | grep main
ffffffff800000f8 T main

Loading the program onto the chip with kflash works fine, but when I open and step through it in GDB, the program usually hits main at either 0x000000ff800000f8 or 0x00000000800000f8. It looks like the program is running and stepping correctly; if I put an infinite loop{}; statement at the beginning of main, the program goes on to stop at ...800000fa. But GDB cannot associate the memory addresses with main because they do not match the records in the ELF file.

Anyone else run into that sort of issue?

[vmedea]

@medusacle Thanks again! This hack with 0xffffffff80000000 address is really great!

happy to hear that !

Using 0xffffffff80000000 for the RAM addresses also seems to work for me, but I think it might cause some issues debugging with GDB and OpenOCD

that's interesting
I cannot check now, but are you sure your code is really executing from 0xffffffff80000000 and not 0x80000000? you wouldn't notice this at all when not using a debugger
it doesn't look like it; the ROM will invoke the code at 0x80000000 and the whole path to main() uses relative addresses

ffffffff80000000 <_start>:
...
ffffffff8000004c:       0060006f                j       ffffffff80000052 <_start_rust>

ffffffff80000052 <_start_rust>:
ffffffff80000052:       1141                    addi    sp,sp,-16
ffffffff80000054:       e406                    sd      ra,8(sp)
ffffffff80000056:       00005097                auipc   ra,0x5
...
ffffffff800000a4:       00001097                auipc   ra,0x1
ffffffff800000a8:       700080e7                jalr    1792(ra) # ffffffff800017a4 <main>

a small change to the startup code could make sure it jumps to the high address…

another less likely cause could be if the upper 32 bits of the pc are hardwired to 0 on the chip so it tells your debugger it's always executing at the low address

[WRansohoff]

Yeah, it does seem like the code is probably running at the 'real' addresses from 0x80000000 on. It would probably be possible to patch the ELF file to remove the 0xFFFFFFFF from the start of each address, but I don't know how to do that properly.

Anyways, it might not be worth digging into too much, because the toolchain should eventually obviate this workaround: rust-lang/rust#59802

[vmedea]

that way around might work as well, got as far as:

riscv64-unknown-elf-objcopy --change-section-address=*-0xffffffff00000000

which moves the ELF sections but doesn't move the DWARF data, which seems to have internal pointers, so isn't useful in your case

[vmedea]

another less likely cause could be if the upper 32 bits of the pc are hardwired to 0 on the chip so it tells your debugger it's always executing at the low address

i checked and this definitely isn't the case,

#![feature(asm)]
#![no_std]
#![no_main]

use core::panic::PanicInfo;
use k210_hal::pac;
use k210_hal::prelude::*;
use k210_hal::stdout::Stdout;
use riscv_rt::entry;

#[panic_handler]
fn panic(info: &PanicInfo) -> ! { loop {} }

extern "C" fn get_pc() -> usize {
    let c: usize;
    unsafe { asm!("auipc $0,0" : "=r"(c) ::: "volatile"); }
    c
}

#[entry]
fn main() -> ! {
    let p = pac::Peripherals::take().unwrap();
    let clocks = k210_hal::clock::Clocks::new();
    let serial = p.UARTHS.constrain(115_200.bps(), &clocks);
    let (mut tx, _) = serial.split();
    let mut stdout = Stdout(&mut tx);
    unsafe {
        writeln!(stdout, "initial pc is 0x{:016x}", get_pc()).unwrap();
        let mut addr = get_pc as usize;
        writeln!(stdout, "function addr is 0x{:016x}", addr).unwrap();
        loop {
            let func: extern "C" fn() -> usize = core::mem::transmute(addr as *const ());
            let pc = func();
            writeln!(stdout, "call 0x{:016x} → pc 0x{:016x}", addr, pc).unwrap();
            addr = addr.wrapping_add(1<<32);
        }
    }
}

gives:

[click for raw output]

initial pc is 0x00000000800003e2
function addr is 0xffffffff8000034c

call 0xffffffff8000034c → pc 0xffffffff8000034c
call 0x000000008000034c → pc 0x000000008000034c
call 0x000000018000034c → pc 0x000000018000034c
call 0x000000028000034c → pc 0x000000028000034c
…
call 0x0000003d8000034c → pc 0x0000003d8000034c
call 0x0000003e8000034c → pc 0x0000003e8000034c
call 0x0000003f8000034c → pc 0x0000003f8000034c
call 0x000000408000034c → pc 0xffffffc08000034c
call 0x000000418000034c → pc 0xffffffc18000034c
call 0x000000428000034c → pc 0xffffffc28000034c
…
call 0x0000007d8000034c → pc 0xfffffffd8000034c
call 0x0000007e8000034c → pc 0xfffffffe8000034c
call 0x0000007f8000034c → pc 0xffffffff8000034c
call 0x000000808000034c → pc 0x000000008000034c
…

• initially, the code runs at the low address 0x000000008xxxxxxx
• after jumping to a high address, pc returns the high address (so doing this once before your code runs will likely solve the debugger issue)
• it looks like bit 38 (0x0000004000000000) is sign-extended, after that it wraps around, so pc is 39 bits wide

[WRansohoff]

Interesting, thanks for the information and taking the time to investigate. That's a neat objcopy trick.

I tried building your example and I see the same thing; it runs from 0x000000008xxxxxxx up to 0x0000003f8xxxxxxx, then from 0xffffffc08xxxxxxx through 0xffffffff8xxxxxxx.

I still haven't gotten GDB to recognize the symbols, but I think there's enough information here to figure it out. Thanks for the example and explanations.

[Disasm]

mcmodel=medium support landed in nightly! rust-lang/rust#62281