Support more FPGA boards

[GiuseppeDiGuglielmo]

Do you plan to support other FPGA boards in addition to the current Genesys 2?

[zarubaf]

Not immediately. But I am happy to accept contributions.

[GiuseppeDiGuglielmo]

Is there any brother/sister board that you may suggest? Is there anything that we should really pay attention at?

[gurkaynak]

There was actually an effort to see if we could use smaller boards like ARTY. Technically (afaik) it would fit, but there is not much more room left in the system after that, and for a development platform it is not ideal, this is why we moved to a larger board. Since we are mainly a research group, we do not have the bandwidth to support multiple boards. As Florian said, we are hoping that the community will provide (and maintain) additional FPGA targets.

[Moschn]

Yes, we have looked at the arty board but it would indeed nearly use up all the space on there.

However, we recently discovered another board that might be a good fit for Ariane: the Avalanche board from Microsemi (https://www.microsemi.com/existing-parts/parts/139680).

• 300k LUTs
• 1gb Ethernet
• 512mb DDR3
• only 180$

Our current flow is based on Vivado so you would have to add a new flow with the Microsemi tools and their IPs which might be quite a bit of work. If you stay with Xilinx based boards the porting effort should be minimal.

[GiuseppeDiGuglielmo]

I was considering two boards indeed for the port:

• Digilent Arty A7 whose chip is 1 order of magnitude smaller than the chip on the Genesys 2
• (or preferably for the reasons that you say) Xilinx Virtex VC707 whose chip is slightly bigger than the Genesys 2

If you think that the port to the VC707 is doable, I may consider a try with some help from you guys (just in case). What do you think?

[zarubaf]

VC707 should be easily possible. We actually had a system running on it in the past. Xilinx, as Moritz said, will allow you to re-use most of our flow. You will need to generate a different mig project file and adapt the constraints file. Definitely willing to help you with issues coming up!

[gurkaynak]

Just wondering, why are you not using Genesys 2? Arty is cheaper I see that, but VC707 is quite an expensive board.

[GiuseppeDiGuglielmo]

Xilinx development boards (like VC707) are really common in academia and usually we get them as a donation through University Programs (in US at least).

I am pretty sure the vc707-porting will be beneficial to many people.

[Iripi97]

@zarubaf or @GiuseppeDiGuglielmo Is there any new update with adding Ariane support for the VC707?

[jmason827]

@GiuseppeDiGuglielmo yes I am also quite interested in this

[jmason827]

@zarubaf hey Florian. we got the thing to work on VC707. not sure how I should go about sharing the changes we made -- what's the best way?

[Iripi97]

@jmason827 Did you only use Vivado tools? If so I would be very interested in knowing how you did!

[jmason827]

@Iripi97 yes, only Vivado tools. note that you'll need a license for Virtex 7 chips. right now I've just finished modifying all the code I could find in the repository to work for the VC707. as it stands, you should be able to make fpga in the root directory.

below is a list of files that were added/modified. the zip contains the new files themselves. replace the old files with the new and you should be able to make fpga for the VC707. let me know if you hit roadbumps.

NOTE: the ariane-for-vc707 BIN file is not in the zip; it wouldn't fit.

ariane_vc707.zip

[zarubaf]

Hey @jmason827 that sounds awesome. So submitting a PR would be a first step, then we can review the code together and I can also give the flow a go on my side. What did you change? One of my concerns is that the flow we provide for the Genesys II is still working otherwise I am more than happy to accept your contributions.

Let me know how that works for you and sorry for the long silence, the thread kind of got overlooked. In case integration should be a problem lets start a new issue.

[jmason827]

@zarubaf copy that. the PR is in.

really the only substantial changes we made were the addition of mig_vc707.prj, some changes to bus widths in ariane_xilinx.sv's port list, and a new .xdc.

we're just happy to potentially contribute -- we're really enjoying using Ariane over here!

[ersin-cukurtas]

I was able to make this work on VC707 with the bit file I got from http://www.princeton.edu/~cloud/openpiton/ and the bbl.bin that I generated using https://github.com/pulp-platform/ariane-sdk repository. I am enjoying using ariane as well so far. I tried to boot debian, but I couldn't. Is it possible to boot debian on vc707?

[Iripi97]

@jmason827 @zarubaf With the new changes to the repository the only addition I had to make on my end for it to work was to add another constraint for 'trst_n' which I set as one of the push buttons similarly to how 'trst' was defined as the center button. It does seem to get caught up when booting the Linux image build with the SDK repository.
It is stuck on the line:
"[ 26.165448 ] lowrisc-digilent-ethernet: Lowrisc ethernet platform (300000000-30007FFF) mapped to ffffffd004028000"
Is there a reason for this? I am going to try the pre-built image to see if the same result occurs.

[Iripi97]

@zarubaf The booting gets stuck on the same line for the pre-built image. Should I move this discussion to the SDK issues section? And is there any way around this holdup?

[jmason827]

@Iripi97 @zarubaf

I had to [...] add another constraint for trst_n

Whoops, this is totally my fault. Meant to change trst in the constraints file back to trst_n.

It does seem to get caught up when booting the Linux image build with the SDK repository.

I should note we did not hook up ethernet correctly. We've not figured out how, and we haven't needed it for our purposes yet.

In any case, I'm able to boot Linux using the build from the SDK repo:

I should probably also note, though, that running Linux seems to get screwy in two cases: 1. Tetris glitches out after a short time playing and 2. the cachetest executable terminates with an Illegal instruction error. This is the wrong thread to go into much detail, but I want to at least disclose these facts.

[zarubaf]

I'd be happy to accept this fixup as a PR.

• The cachetest is currently ment to fail as we do not support delegating the performance counters to user mode. So access to the CSR will just trap with an illegal instruction. It was a bug before v4.2 that made these registers accessible to user mode.
• We have seen similar things on the Tetris side. Are you using the UART which we provide? I'd suspect some problems with it. But it could also be the core of course.

[Iripi97]

@jmason827 Do you think you could post your bbl.bin file? I would like to check to see if it was maybe an issue on my end with generating the boot image which is causing it to hang up at the "lowrisc-digilent-ethernet: Lowrisc ethernet platform (30000000-30007FFF) mapped to …" line.

[jmason827]

@Iripi97 this is the one I've been using (zipped because couldn't attach *.bin file):

bbl.zip

[Iripi97]

@jmason827 Do you think you could do me one more favor please? Would it be possible for you to zip together your .bit , .mcs , and .prm files you used when you received the (full) output you posted earlier? We are trying to verify if it is a software or hardware issue on our end. Any response would be greatly appreciated!

[lukasauer]

I fixed the boot process on the VC707 by removing the ethernet node from the device tree (fpga/src/bootrom/ariane.dts). Regenerate the bootrom afterwards by running make all inside fpga/.

Regarding ethernet on the VC707. The PHY is connected to the FPGA using the SGMII interface. To use the ethernet core from Ariane, a SGMII to RGMII converter would be needed. I am not sure how much effort it would take to develop.

[RaphaelKlink]

There is another option you could try.
I was unfortunaly not able you get this option running on an VCU108 Board.
The ethernet is split in two parts:

MAC (Media Access Controller) (AXI to GMII)

GMII to RGMII Bridge

Xilinx has a GMII to SGMII bridge IP Core
https://www.xilinx.com/support/documentation/ip_documentation/gig_ethernet_pcs_pma/v16_1/pg047-gig-eth-pcs-pma.pdf

(Xilinx also offers a GMII to RGMII Bridge IP Core, currently only for the Zync family
https://www.xilinx.com/support/documentation/ip_documentation/gmii_to_rgmii/v4_0/pg160-gmii-to-rgmii.pdf)

I tried to exchange the GMII to RGMII Bridge with the GMII to SGMII Bridge but was unsuccesfull so far.

[Iripi97]

@lukasauer I have tried modifying my 'ariane.dts' file to get the boot run to complete but it is still getting stuck on the "lowrisc-digilent-ethernet: Lowrisc ethernet platform …". Are you sure changing this is what fixed the boot for your VC707.

[lukasauer]

Make sure to also regenerate the bootrom after editing the device tree. You should see changes to fpga/src/bootrom/bootrom.h and fpga/src/bootrom/bootrom.sv afterwards.

[lukasauer]

@RaphaelKlink Good point, that should work. Please let us know if you get it working, that would be really helpful! :)

[Iripi97]

@lukasauer I'm slightly confused, whenever I execute 'make all' inside the 'fpga' directory it just begins the process of generating the bit-stream and MCS file with Vivado, it does not remake the bootrom. Is this changing of bootrom happening implicitly or should I be going to another directory to ensure this is happening?

[lukasauer]

You are right, I didn't remember the directory correctly. make all should be run from fpga/src/bootrom/. This will just rebuild the bootrom, you will have to rebuild the bit file afterwards. Also make sure that you have a RISC-V toolchain available.

[yge123]

@lukasauer @Iripi97

I could not find ethernet node from the device tree (fpga/src/bootrom/ariane.dts),
so removed eth from compatible = lines, and make all to update bootrom.h and bootrom.sv
and then regenerated bit stream. but boot process still stuch at "[ 26.165448 ] lowrisc-digilent-ethernet: Lowrisc ethernet platform (300000000-30007FFF) mapped to ffffffd004028000"

what should I do to make it right?

< compatible = "ariane-bare-dev";
< model = "ariane-bare";

compatible = "eth,ariane-bare-dev";
model = "eth,ariane-bare";
17c17
< compatible = "ariane", "riscv";

---

  compatible = "eth, ariane", "riscv";

36c36
< compatible = "ariane-bare-soc", "simple-bus";

compatible = "eth,ariane-bare-soc", "simple-bus";

[xushengj]

@yge123 that's not the correct way to remove ethernet node. you should find the node to be from line 111 to line 118 in fpga/src/bootrom/ariane.dts:

    eth: lowrisc-eth@30000000 {
      compatible = "lowrisc-eth";
      device_type = "network";
      interrupt-parent = <&PLIC0>;
      interrupts = <3 0>;
      local-mac-address = [00 18 3e 02 e3 7f]; // This needs to change if more than one GenesysII on a VLAN
      reg = <0x0 0x30000000 0x0 0x8000>;
    };

add "//" at start of each line to comment them out, like how mmc-slot is commented out right above it.

[yge123]

@xushengj it already doesn't have eth: section ,line 111 to 118. it only has 72 lines
but it has the same booting issue.

[xushengj]

@yge123 You are probably looking at "bootrom/ariane.dts" instead of "fpga/src/bootrom/ariane.dts". I made the same mistake a few days ago. I think the first one is the bootrom for simulation and the second one is for FPGAs.

[yge123]

@xushengj @Iripi97 @lukasauer

can you share your local dts file? it seems that the latest repo doesn't have eth: node but has the same boot issue.

[yge123]

@xushengj thank you very much for catching my mistake.

[xushengj]

@zarubaf I made a pull request for adding support for Nexys Video board. It has an Artix-7 FPGA and everyone can use Vivado WebPACK edition to run Ariane on it. However I did not get everything working properly; Ethernet is not working yet and there is some issue in UART. Any help is appreciated..

[AlexZeh-Crypto]

Hello,

what was the highest clock rate achieved for the ariane core on a FPGA?

[ersin-cukurtas]

@yge123 that's not the correct way to remove ethernet node. you should find the node to be from line 111 to line 118 in fpga/src/bootrom/ariane.dts:

    eth: lowrisc-eth@30000000 {
      compatible = "lowrisc-eth";
      device_type = "network";
      interrupt-parent = <&PLIC0>;
      interrupts = <3 0>;
      local-mac-address = [00 18 3e 02 e3 7f]; // This needs to change if more than one GenesysII on a VLAN
      reg = <0x0 0x30000000 0x0 0x8000>;
    };

add "//" at start of each line to comment them out, like how mmc-slot is commented out right above it.
@xushengj I did what you suggested, and ran a clean build. It still gets stuck at the same place. Did this work for you?

[olivetreezhao]

@Iripi97 yes, only Vivado tools. note that you'll need a license for Virtex 7 chips. right now I've just finished modifying all the code I could find in the repository to work for the VC707. as it stands, you should be able to make fpga in the root directory.

below is a list of files that were added/modified. the zip contains the new files themselves. replace the old files with the new and you should be able to make fpga for the VC707. let me know if you hit roadbumps.

NOTE: the ariane-for-vc707 BIN file is not in the zip; it wouldn't fit.

ariane_vc707.zip

Hi,
I am trying to port the ariane to the Xilinx ZCU102 board that has the Zynq UltraScale XCZU9EG device. I am trying to generate the mig ddr4 in Vivado, but I did not have the .prj file generated. Just wondering how you generated the mig_vc707.prj? I use Vivado 2018.3.

Thanks!

[RaphaelKlink]

I have got the vcu 108 working.
the ddr4 controller for xilinx is completely different from the ddr3 controller.
It is much easier for you to create a blockdesign. Then there is a board tab. Choose a ddr4 channel and let vivado automatically create the controller.
Afterwards you wrap the whole controller_bd.
Then you need insert the axi dwidth512_64 converter between the controller and the clock converter.
Also you need to check the output clock of the ddr4 controller. It is used as the input clock for the clocking wizard for the core.

I hope I am finished with the correct makefile for vcu108 this week

[olivetreezhao]

I have got the vcu 108 working.
the ddr4 controller for xilinx is completely different from the ddr3 controller.
It is much easier for you to create a blockdesign. Then there is a board tab. Choose a ddr4 channel and let vivado automatically create the controller.
Afterwards you wrap the whole controller_bd.
Then you need insert the axi dwidth512_64 converter between the controller and the clock converter.
Also you need to check the output clock of the ddr4 controller. It is used as the input clock for the clocking wizard for the core.

I hope I am finished with the correct makefile for vcu108 this week

Thanks so much for the information! I still have a few questions:

1. Should I wrap the ddr4 controller separately from the data width converter and the clock converter or wrap them together as an IP block?
2. Why the data width converter and the clock converter are required? (I am not quite familiar with clocking yet.)
3. Would you be able to share the screenshot of your block design?
4. After wrapping the project, how to merge it with the rest of the ariane platform?
   Thanks!

[RaphaelKlink]

https://github.com/RaphaelKlink/ariane

Here is my current version. It should be working. But it is only working for the vcu 108. I am currently not able to choose between the ddr3 and ddr4 controller.

@zarubaf do you have an idea how to solve this problem
https://github.com/RaphaelKlink/ariane/blob/master/fpga/Makefile

[olivetreezhao]

https://github.com/RaphaelKlink/ariane

Here is my current version. It should be working. But it is only working for the vcu 108. I am currently not able to choose between the ddr3 and ddr4 controller.

@zarubaf do you have an idea how to solve this problem
https://github.com/RaphaelKlink/ariane/blob/master/fpga/Makefile

Thanks for sharing!

[leemango1998]

@Iripi97 yes, only Vivado tools. note that you'll need a license for Virtex 7 chips. right now I've just finished modifying all the code I could find in the repository to work for the VC707. as it stands, you should be able to make fpga in the root directory.
below is a list of files that were added/modified. the zip contains the new files themselves. replace the old files with the new and you should be able to make fpga for the VC707. let me know if you hit roadbumps.
NOTE: the ariane-for-vc707 BIN file is not in the zip; it wouldn't fit.

ariane_vc707.zip

Hi,
I am trying to port the ariane to the Xilinx ZCU102 board that has the Zynq UltraScale XCZU9EG device. I am trying to generate the mig ddr4 in Vivado, but I did not have the .prj file generated. Just wondering how you generated the mig_vc707.prj? I use Vivado 2018.3.

Thanks!

Hey, olivetreezhao！I am also trying to port the ariane to zcu106. And i have succeeded in generating the bitstream. However, i meet trouble in generating the ariane.mcs file. Thus it can't boot from the QSPI flash. Any help is appreciated. You can contact me by email: [email protected], if it doesn't bother you. Thanks !

[RaphaelKlink]

@leemango1998
First: You can simply program the zcu106 directly through the vivado interface with the bitstream. You do not necessarily need to create the mcs file.

Secondly: Could you describe the problem in more detail(vivado logs, etc), so we can help you better

[leemango1998]

@RaphaelKlink Very glad to receive your reply. I am a beginner in FPGA, so some of my problems may be weird. Sorry about that. Actually i want to know that although the bitstream is downloaded into the zcu106, nothing happened. I don't know whether the various parts of the SoC(such as DDR4, UART) work properly on zcu106. So how to ensure that all work properly on FPGA. Furthermore, if i want to run a C program in ariane SoC on FPGA and print the results via UART, what should i do?

[RaphaelKlink]

No Problem,

for your first question. I can not really help you, unfortunately. Without the FPGA Board debugging is nearly impossible.
I can only give you some general tips. First, check the constraint file. Make sure every Input and Output has the correct Package PIN and IO Standard. The second thing is the DDR4 controller. Recreate it with the steps i described in a previous post.

For your second question. If you want to use the UART as an Output you need to be able to boot Linux. As the UART driver is only available if you execute the application you want in an OS environment.

[Moschn]

Just a short sidenote about debugging on the FPGA: integrated logix analyzers (ILA) help a fair bit. They can be instantiated in the core and can be used to get a reading on internal signals with a trigger. Comparable to an oscilloscope. This repo already contains an example ILA instantiation in the commit stage (commented out) and a corresponding Xilinx IP tcl + Makefile.

Wit an ILA in the commit stage, you should be able to observe the addresses of committed instructions. If, e.g., your DDR does not work, then I expect this instruction trace to stall pretty quickly.

[jake-ke]

Hello, @jmason827 , thank you for sharing your VC707 implementation. I saw the FT2232 (JTAG) pins in the constraints are connected to the FMC pins. I guess you have made FT2232 working on VC707 through FMC. I need to do the same for my project. Could you please share how you do that? Thanks!

cva6/fpga/constraints/vc707.xdc

Lines 4 to 9 in d24287e

	## To use FTDI FT2232 JTAG
	set_property -dict { PACKAGE_PIN AV39 IOSTANDARD LVCMOS18 } [get_ports trst];
	set_property -dict { PACKAGE_PIN M32 IOSTANDARD LVCMOS18 } [get_ports tck ];
	set_property -dict { PACKAGE_PIN V29 IOSTANDARD LVCMOS18 } [get_ports tdi ];
	set_property -dict { PACKAGE_PIN M28 IOSTANDARD LVCMOS18 } [get_ports tdo ];
	set_property -dict { PACKAGE_PIN U31 IOSTANDARD LVCMOS18 } [get_ports tms ];

[farnamatic]

@Iripi97 yes, only Vivado tools. note that you'll need a license for Virtex 7 chips. right now I've just finished modifying all the code I could find in the repository to work for the VC707. as it stands, you should be able to make fpga in the root directory.
below is a list of files that were added/modified. the zip contains the new files themselves. replace the old files with the new and you should be able to make fpga for the VC707. let me know if you hit roadbumps.
NOTE: the ariane-for-vc707 BIN file is not in the zip; it wouldn't fit.

ariane_vc707.zip

Hi, I am trying to port the ariane to the Xilinx ZCU102 board that has the Zynq UltraScale XCZU9EG device. I am trying to generate the mig ddr4 in Vivado, but I did not have the .prj file generated. Just wondering how you generated the mig_vc707.prj? I use Vivado 2018.3.

Thanks!

Hi @olivetreezhao, did you succeed in porting the Ariane from the Genesys to the Zynq platform?
I intend to do a similar job but, I also want to use the PS DDR (not PL DDR). How have you managed to connect the ariane_peripherals (like SD SPI) to the PS? Because the SD card of the ZU platforms is connected to the SDIO or MIO. Maybe there are some solutions to use EMIO and have access to an SD card connected to the MIO banks of the PS! (possibly)

If anybody has some effort in porting the Ariane from the Genesys to the Zynq (or ZU+), please help me. I would appreciate it.

Thanks!

[olivetreezhao]

@farnam16 No, I did not continue with the porting work eventually.

[farnamatic]

@farnam16 No, I did not continue with the porting work eventually.

@olivetreezhao Thanks for your prompt reply. mm, why? is it impossible you mean? or there are other reasons you stopped working on it?

[olivetreezhao]

Originally I thought it would be a straightforward porting, then I had a tech issue (could not remember what it was though), and the porting work does not have priority, I have to give up to work on something else. Wish you good luck on porting it!

[farnamatic]

Originally I thought it would be a straightforward porting, then I had a tech issue (could not remember what it was though), and the porting work does not have priority, I have to give up to work on something else. Wish you good luck on porting it!

Ah! thanks again. I am afraid if it requires a lot of effort. There are several questions to be answered for porting the Ariane from the Genesys to the ZU+ board.

1. How we load the binary file of our application (for example generated binary of Hello_world.c) to be executed on the Ariane RISCV (in PL).

2. Where we store our root file system. If in the SD card, how do we provide access to the SDIO from the PL? How do we provide access to the UART (for serial terminal) which is connected to the PS peripherals?.

3. In my case, we do not need MIG, because we pass through the PS AXI Slave port to the DDR4. There are also modifications that should be applied on the HW modules and ariane.dts.