From ade8457870fdf12e8dfd3f47b1a35f23fc634221 Mon Sep 17 00:00:00 2001 From: Abraham Gonzalez Date: Tue, 9 Mar 2021 05:11:24 +0000 Subject: [PATCH] First doc pass (no updated imgs) [ci skip] --- docs/Advanced-Concepts/Chip-Communication.rst | 134 +++++++++++++----- docs/Advanced-Concepts/Harness-Clocks.rst | 20 +++ docs/Advanced-Concepts/index.rst | 1 + .../src/main/scala/HarnessBinders.scala | 2 + .../src/main/scala/config/RocketConfigs.scala | 2 + 5 files changed, 120 insertions(+), 39 deletions(-) create mode 100644 docs/Advanced-Concepts/Harness-Clocks.rst diff --git a/docs/Advanced-Concepts/Chip-Communication.rst b/docs/Advanced-Concepts/Chip-Communication.rst index 50c5ac9a..9084f7ef 100644 --- a/docs/Advanced-Concepts/Chip-Communication.rst +++ b/docs/Advanced-Concepts/Chip-Communication.rst @@ -7,7 +7,7 @@ There are two types of DUTs that can be made: `tethered` or `standalone` DUTs. A `tethered` DUT is where a host computer (or just host) must send transactions to the DUT to bringup a program. This differs from a `standalone` DUT that can bringup itself (has its own bootrom, loads programs itself, etc). An example of a tethered DUT is a Chipyard simulation where the host loads the test program into the DUTs memory and signals to the DUT that the program is ready to run. -An example of a standalone DUT is a Chipyard simulation where a program can be loaded from an SDCard by default. +An example of a standalone DUT is a Chipyard simulation where a program can be loaded from an SDCard out of reset. In this section, we mainly describe how to communicate to tethered DUTs. There are two ways the host (otherwise known as the outside world) can communicate with a tethered Chipyard DUT: @@ -45,33 +45,21 @@ Using the Tethered Serial Interface (TSI) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ By default, Chipyard uses the Tethered Serial Interface (TSI) to communicate with the DUT. -TSI protocol is an implementation of HTIF that is used to send commands to the -RISC-V DUT. These TSI commands are simple R/W commands -that are able to probe the DUT's memory space. During simulation, the host sends TSI commands to a -simulation stub called ``SimSerial`` (C++ class) that resides in a ``SimSerial`` Verilog module -(both are located in the ``generators/testchipip`` project). This ``SimSerial`` Verilog module then -sends the TSI command recieved by the simulation stub into the DUT which then converts the TSI -command into a TileLink request. This conversion is done by the ``SerialAdapter`` module -(located in the ``generators/testchipip`` project). In simulation, FESVR -resets the DUT, writes into memory the test program, and indicates to the DUT to start the program -through an interrupt (see :ref:`customization/Boot-Process:Chipyard Boot Process`). Using TSI is currently the fastest -mechanism to communicate with the DUT in simulation. - -In the case of a chip tapeout bringup, TSI commands can be sent over a custom communication -medium to communicate with the chip. For example, some Berkeley tapeouts have a FPGA -with a RISC-V soft-core that runs FESVR. The FESVR on the soft-core sends TSI commands -to a TSI-to-TileLink converter living on the FPGA (i.e. ``SerialAdapter``). After the transaction is -converted to TileLink, the ``TLSerdesser`` (located in ``generators/testchipip``) serializes the -transaction and sends it to the chip (this ``TLSerdesser`` is sometimes also referred to as a -serial-link or serdes). Once the serialized transaction is received on the -chip, it is deserialized and masters a bus on the chip. The following image shows this flow: - -.. image:: ../_static/images/chip-bringup.png - -.. note:: - The ``TLSerdesser`` can also be used as a slave (client), so it can sink memory requests from the chip - and connect to off-chip backing memory. Or in other words, ``TLSerdesser`` creates a bi-directional TileLink - interface. +TSI protocol is an implementation of HTIF that is used to send commands to the RISC-V DUT. +These TSI commands are simple R/W commands that are able to probe the DUT's memory space. +During simulation, the host sends TSI commands to a simulation stub in the test harness called ``SimSerial`` +(C++ class) that resides in a ``SimSerial`` Verilog module (both are located in the ``generators/testchipip`` +project). +This ``SimSerial`` Verilog module then sends the TSI command recieved by the simulation stub +to an adapter that converts the TSI command into a TileLink request. +This conversion is done by the ``SerialAdapter`` module (located in the ``generators/testchipip`` project). +After the transaction is converted to TileLink, the ``TLSerdesser`` (located in ``generators/testchipip``) serializes the +transaction and sends it to the chip (this ``TLSerdesser`` is sometimes also referred to as a digital serial-link or SerDes). +Once the serialized transaction is received on the chip, it is deserialized and masters a TileLink bus on the chip +which handles the request. +In simulation, FESVR resets the DUT, writes into memory the test program, and indicates to the DUT to start the program +through an interrupt (see :ref:`customization/Boot-Process:Chipyard Boot Process`). +Using TSI is currently the fastest mechanism to communicate with the DUT in simulation and is also used by FireSim. Using the Debug Module Interface (DMI) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -95,9 +83,10 @@ Thus, Chipyard removes the DTM by default in favor of the TSI protocol for DUT c Starting the TSI or DMI Simulation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -All default Chipyard configurations use TSI to communicate between the simulation and the simulated SoC/DUT. Hence, when running a -software RTL simulation, as is indicated in the :ref:`simulation/Software-RTL-Simulation:Software RTL Simulation` section, you are in-fact using TSI to communicate with the DUT. As a -reminder, to run a software RTL simulation, run: +All default Chipyard configurations use TSI to communicate between the simulation and the simulated SoC/DUT. +Hence, when running a software RTL simulation, as is indicated in the +:ref:`simulation/Software-RTL-Simulation:Software RTL Simulation` section, you are in-fact using TSI to communicate with the DUT. +As a reminder, to run a software RTL simulation, run: .. code-block:: bash @@ -105,11 +94,10 @@ reminder, to run a software RTL simulation, run: # or cd sims/vcs - make CONFIG=LargeBoomConfig run-asm-tests + make CONFIG=RocketConfig run-asm-tests -FireSim FPGA-accelerated simulations use TSI by default as well. - -If you would like to build and simulate a Chipyard configuration with a DTM configured for DMI communication, then you must tie-off the TSI interface, and instantiate the `SimDTM`. Note that we use `WithTiedOffSerial ++ WithSimDebug` instead of `WithTiedOffDebug ++ WithSimSerial`. +If you would like to build and simulate a Chipyard configuration with a DTM configured for DMI communication, +then you must tie-off the serial-link interface, and instantiate the `SimDTM`. .. literalinclude:: ../../generators/chipyard/src/main/scala/config/RocketConfigs.scala :language: scala @@ -130,13 +118,81 @@ Using the JTAG Interface ------------------------ The main way to use JTAG with a Rocket Chip based system is to instantiate the Debug Transfer Module (DTM) -and configure it to use a JTAG interface. The default Chipyard designs instantiate the DTM and configure it -to use JTAG. You may attach OpenOCD and GDB to any of the default JTAG-enabled designs. +and configure it to use a JTAG interface. +The default Chipyard designs instantiate the DTM and configure it to use JTAG. +You may attach OpenOCD and GDB to any of the default JTAG-enabled designs. Debugging with JTAG -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~ -Please refer to the following resources on how to debug with JTAG. +Please refer to the following resources on how to debug a Rocket Chip system with JTAG. * https://github.com/chipsalliance/rocket-chip#-debugging-with-gdb * https://github.com/riscv/riscv-isa-sim#debugging-with-gdb + +Roughly the steps are as follows (refer to the links for details): + +1. Build a Chipyard JTAG-enabled RTL design (i.e. ``make CONFIG=RocketConfig`` in ``sims/*``) +2. Run the simulation with remote bit-bang enabled (i.e. ``make CONFIG=RocketConfig BINARY= SIM_FLAGS="+jtag_rbb_enable=1 --rbb-port=9823" run-binary``) +3. Launch OpenOCD +4. Connect to OpenOCD via GDB +5. Done! + +Example Test Chip Bringup Communication +--------------------------------------- + +Intro to Typical Chipyard Test Chip +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Most, if not all, Chipyard configurations are tethered using TSI (over a serial link) and have access +to external memory through an AXI port (backing AXI memory). +The following image shows the DUT with these set of default signals: + +.. image:: ../_static/images/chip-bringup.png + +In this setup, the serial-link is connected to the TSI/FESVR peripherals while the AXI port is connected +to a simulated AXI memory. +However, AXI ports tend to have many signals associated with them so instead of creating an AXI port off the DUT, +one can send the memory transactions over the bi-directional serial-link (``TLSerdesser``) so that main +interface to the DUT is the serial-link (which as comparatively less signals than an AXI port). +This new setup (shown below) is a typical Chipyard test chip setup: + +.. image:: ../_static/images/chip-bringup.png + +Simulation Setup +~~~~~~~~~~~~~~~~ + +To test this type of configuration (TSI/memory transactions over the serial-link), most of the same TSI collateral +would be used. +The main difference is that the TileLink-to-AXI converters and simulated AXI memory resides on the other side of the +serial-link. + +.. image:: ../_static/images/chip-bringup.png + +.. note:: + Here the simulated AXI memory and the converters can be in a different clock domain in the test harness + than the DUT. This is done in a harness binder doing these offchip connections. See + :ref:`Advanced-Concepts/Harness-Clocks:Creating Clocks in the Test Harness` on how to generate a clock in a harness binder. + +This type of simulation setup is done in the following multi-clock configuration: + +.. literalinclude:: ../../generators/chipyard/src/main/scala/config/RocketConfigs.scala + :language: scala + :start-after: DOC include start: MulticlockAXIOverSerialConfig + :end-before: DOC include end: MulticlockAXIOverSerialConfig + +Real-world FPGA Setup +~~~~~~~~~~~~~~~~~~~~~ + +Assuming this same chip configuration is being tested after tapeout (during bringup), +communication with the DUT is similar but slightly different. +For example, some Berkeley tapeouts have a FPGA with a RISC-V soft-core that runs FESVR. +This RISC-V soft-core would serve as the host of the test that will run on the DUT. +The FESVR on the soft-core sends TSI commands to the ``SerialAdapter`` which then sends the converted +TileLink transaction to the chip over the serial-link. +If the chip requests offchip memory, it can then send the transaction back over the serial-link to the +FPGA DRAM. +The following image shows this flow: + +.. image:: ../_static/images/chip-bringup.png + diff --git a/docs/Advanced-Concepts/Harness-Clocks.rst b/docs/Advanced-Concepts/Harness-Clocks.rst new file mode 100644 index 00000000..e0a7ac3e --- /dev/null +++ b/docs/Advanced-Concepts/Harness-Clocks.rst @@ -0,0 +1,20 @@ +.. _harness-clocks: + +Creating Clocks in the Test Harness +=================================== + +By default, all modules in the Test Harness, including those made by harness binders +use the implicit clock and reset of the Test Harness. +However, the test harness and harness binders, have the ability to generate a standalone +clock and reset signal. +This is done by the ``HarnessClockInstantiator`` which allows you to request a clock at a +particular frequency. +Take the following harness binder example: + +.. literalinclude:: ../../generators/chipyard/src/main/scala/config/HarnessBinders.scala + :language: scala + :start-after: DOC include start: HarnessClockInstantiatorEx + :end-before: DOC include end: HarnessClockInstantiatorEx + +While the purpose of the binder isn't necessary here, you can see that the ``p(HarnessClockInstantiatorKey).getClockBundle`` +allows the binder to request a clock/reset bundle at a particular frequency. diff --git a/docs/Advanced-Concepts/index.rst b/docs/Advanced-Concepts/index.rst index 12b12716..b294d11b 100644 --- a/docs/Advanced-Concepts/index.rst +++ b/docs/Advanced-Concepts/index.rst @@ -14,4 +14,5 @@ They expect you to know about Chisel, Parameters, configs, etc. Debugging-BOOM Resources CDEs + Harness-Clocks diff --git a/generators/chipyard/src/main/scala/HarnessBinders.scala b/generators/chipyard/src/main/scala/HarnessBinders.scala index a1f39c6c..bed779d0 100644 --- a/generators/chipyard/src/main/scala/HarnessBinders.scala +++ b/generators/chipyard/src/main/scala/HarnessBinders.scala @@ -157,11 +157,13 @@ class WithSimAXIMemOverSerialTL extends OverrideHarnessBinder({ } ports.map({ port => +// DOC include start: HarnessClockInstantiatorEx val harnessMultiClockAXIRAM = SerialAdapter.connectHarnessMultiClockAXIRAM( system.serdesser.get, port, p(HarnessClockInstantiatorKey).getClockBundle("mem_over_serial_tl_clock", memFreq), th.harnessReset) +// DOC include end: HarnessClockInstantiatorEx val success = SerialAdapter.connectSimSerial(harnessMultiClockAXIRAM.module.io.tsi_ser, port.clock, th.harnessReset.asBool) when (success) { th.success := true.B } diff --git a/generators/chipyard/src/main/scala/config/RocketConfigs.scala b/generators/chipyard/src/main/scala/config/RocketConfigs.scala index 45f016bf..a81c1449 100644 --- a/generators/chipyard/src/main/scala/config/RocketConfigs.scala +++ b/generators/chipyard/src/main/scala/config/RocketConfigs.scala @@ -213,6 +213,7 @@ class LBWIFRocketConfig extends Config( new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new chipyard.config.AbstractConfig) +// DOC include start: MulticlockAXIOverSerialConfig class MulticlockAXIOverSerialConfig extends Config( new chipyard.config.WithSystemBusFrequencyAsDefault ++ new chipyard.config.WithSystemBusFrequency(250) ++ @@ -233,3 +234,4 @@ class MulticlockAXIOverSerialConfig extends Config( new freechips.rocketchip.subsystem.WithNBigCores(2) ++ new chipyard.config.AbstractConfig) +// DOC include end: MulticlockAXIOverSerialConfig