Rust on Dreamcast

From dreamcast.wiki
Revision as of 18:00, 25 November 2024 by Darc (talk | contribs) (→‎Installing our custom KallistiOS and GCC toolchain)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Ferris holding his Dreamcast controller

Rust is a popular modern programming language focusing on performance, memory safety, and program correctness. As a "blazingly fast" low-level language, it is viable for running on a Dreamcast console. Doing so presents a bit of a challenge, however, as the official Rust compiler is based on the LLVM toolchain infrastructure, which does not support the Dreamcast CPU's SuperH architecture. Dreamcast programming is instead typically done with GCC, the GNU Compiler Collection. There are currently two viable solutions to this challenge:

  • rustc_codegen_gcc, the current preferred method: An experimental codegen backend for the official Rust rustc compiler which interfaces with GCC's libgccjit API
  • gccrs: an experimental Rust frontend for GCC

Neither solution is complete at this time, but rustc_codegen_gcc is much further along and is quite usable with some patience with its current limitations. On the other hand, gccrs can compile for Dreamcast, but is in an earlier state. Below we will focus on using rustc_codegen_gcc. For more information on using gccrs, see the gccrs page.

Current Status

  • The full std library is supported.
  • A custom version of KallistiOS v2.1.0 (latest stable release) is used.
    • This version includes minor changes for Rust and a beta libpthread addon layer to interface with the Rust standard library.
    • Wrappers are provided to invoke cargo and rustc in a familiar manner to compile projects.
  • A kos-sys raw/unsafe bindings crate is provided for easily linking KallistiOS and its unsafe C API into Rust crates. This crate is in active development, but with a large portion of KallistiOS functionality available now via C FFI.
  • A kos-rs Rust crate with safe idiomatic interfaces is available but is in very early stages.
  • Code generation uses rustc_codegen_gcc, so its limitations apply here. See the development blog for more information on its progress.
    • rustc_codegen_gcc is pinned to a particular Rust nightly version, but is synced regularly. The current version in use is the 2024-08-10 nightly.
    • A KallistiOS-patched version of the latest nightly is maintained and updated regularly.
    • Panic unwinding, debug info, LTO, etc. are not currently available.
    • Architectures dependent on rustc_codegen_gcc are not yet able to be added to the rustc frontend, so a workaround is necessary: MIPS is falsely selected for KallistiOS, causing rustc to emit objects with a MIPS header and a provided wrapper is used when linking to rewrite these MIPS headers to SuperH before passing the compiled objects to the linker.
  • A separate development version of the GCC toolchain must be used.
    • This version of GCC contains the latest libgccjit patches necessary to interface with rustc_codegen_gcc, and will be built to work with the custom KallistiOS's libpthread layer.
    • New updates to rustc_codegen_gcc often require new updates to GCC, so recompiling GCC regularly will be necessary to keep up with the latest updates.
    • The development version of GCC does not always compile well on every platform. Using a modern Linux platform is recommended for maximum compatibility.
    • The custom KallistiOS and GCC environment can still be used to compile code or projects written in C, C++, etc.
  • KallistiOS and all related libraries and tools must be compiled using the -m4-single ABI. This is already set up for you in the guide below, but if external libraries are being linked to Rust projects, make sure that code is compiled using the proper ABI.

Setting up a Rust environment for Dreamcast development

The following guide will provide instructions for setting up a Rust development environment for Dreamcast.

If you run into any errors or other challenges while following this tutorial, or simply need clarification on any of the steps, feel free to ask for assistance on the message board or our Discord server and we would be happy to aid you and update the guide for the benefit of future readers and others in the community.

Prerequisites

First, we will need to set up the prerequisites.

  • You must have the rustup tool installed for your operating system to manage Rust toolchain installations. It will automatically manage the latest Rust nightly installations for you. You cannot use a Rust toolchain that may be provided by your operating system vendor.
  • Install the dependency packages for your operating system. These are listed in the Dependencies section on the Getting Started with Dreamcast development#Dependencies page, with lists provided for most common operating systems.
  • We will install our custom Rust toolchain environment within /opt/toolchains/dc/rust. If it does not already exist, create the /opt/toolchains/dc directory and grant it proper permissions using the following commands.
sudo mkdir -p /opt/toolchains/dc
sudo chmod -R 755 /opt/toolchains/dc
sudo chown -R $(id -u):$(id -g) /opt/toolchains/dc
  • Clone the Rust for Dreamcast repo containing necessary support files to /opt/toolchains/dc/rust:
git clone https://github.com/dreamcast-rs/rust-for-dreamcast.git /opt/toolchains/dc/rust

Installing our custom KallistiOS and GCC toolchain

Clone the git repository for our custom Rust-patched version of KallistiOS v2.1.0 stable:

git clone https://github.com/dreamcast-rs/KallistiOS /opt/toolchains/dc/rust/kos

Enter the dc-chain directory:

cd /opt/toolchains/dc/rust/kos/utils/dc-chain

The Makefile.default.cfg file in this directory has been customized for you, selecting the proper toolchain profile, paths, and build options necessary for building a Rust toolchain using this guide. It is not recommended to adjust the defaults.

To build the GCC toolchain using 2 CPU cores, simply type:

make makejobs=2 sh_gcc_git_branch=master-88b2da3c175d34d65415f69961420f35ac1a1c56

Adjust the makejobs=... value to use the number of CPU cores available to you on your system in order to speed up compilation. This may take a while to build, so be patient.

Once the GCC toolchain is built, source the environ.sh KallistiOS environment settings. A custom environ.sh file has been provided for you, containing settings pre-adjusted for Rust development using this guide.

You will need to run the source command to apply the KallistiOS environment settings to your currently running shell. Run the following now, and whenever you open a new shell to work on Dreamcast projects:

source /opt/toolchains/dc/rust/misc/environ.sh

Enter the KallistiOS directory:

cd /opt/toolchains/dc/rust/kos

Build KallistiOS:

make

Building rustc_codegen_gcc and Rust sysroot

Now that we have a working KallistiOS environment suitable for Rust development set up, we can build the Rust compiler compiler and sysroot components. A script is provided which will download the necessary components from the respective repositories and compile them for you. Run the installer script:

/opt/toolchains/dc/rust/misc/install-rust.sh

If all goes well, you'll see Rust for KallistiOS/Dreamcast installed! message!

Building KOS ports libraries

If desired, you may also built the KOS ports collection of libraries for use with your Rust environment. It is recommended to maintain a separate KOS ports installation for linking with Rust projects in /opt/toolchains/dc/rust/kos-ports. This is because the Rust environment is using the -m4-single ABI, but the KallistiOS default installation will be using the -m4-single-only, which cannot be mixed and matched.

Clone the kos-ports repository to your system:

git clone https://github.com/KallistiOS/kos-ports /opt/toolchains/dc/rust/kos-ports

An individual port can be built and installed by entering its directory and running the proper command. For example, installing GLdc:

cd /opt/toolchains/dc/rust/kos-ports/libGL
make install

Or, instead, you may run the script to build all of the included ports:

/opt/toolchains/dc/rust/kos-ports/utils/build-all.sh

Using Rust for Dreamcast

If all went well, you will now have a working Rust for Dreamcast environment!

  • cargo can be invoked to target Dreamcast using the kos-cargo command.
    • kos-cargo run can be used to build and run code on a Dreamcast console using dcload-ip or dcload-serial. You will need to set up the KOS_LOADER variable with the necessary command in the /opt/toolchains/dc/rust/misc/environ.sh file (and then re-source the file for the change to take effect).
  • rustc can be invoked to target Dreamcast using the kos-rustc command.

Examples

Examples are included with the Rust for Dreamcast repo to help you get started.

  • hello-cargo demonstrates how to create a simple "Hello, world!" application using cargo.
  • hello-make demonstrates how to create a simple "Hello, world!" application using standard Makefiles.
  • gldc-cube demonstrates a Rust project using the GLdc KOS port.
  • addlib demonstrates how to create a Rust library that can be included with a KallistiOS project.
  • atomics demonstrates the use of atomic functions.
  • filesystem-io demonstrates listing directory contents and opening files.
  • network-time demonstrates displaying the system time and retrieving the current UTC time from an NTP server using the Broadband adapter or LAN adapter.
  • pvr_strips_direct demonstrates the use of PVR direct rendering
  • threads demonstrates the use of threads.
  • tokio-async demonstrates the use of the tokio async runtime.

Creating a new Rust project with Cargo

First, we'll demonstrate creating a new "Hello, world!" project with kos-cargo. This will follow the cargo-hello example included in the Rust-for-Dreamcast repo.

In a directory of your choosing, let's invoke kos-cargo to create a new project and then enter the directory:

kos-cargo new hello
cd hello

Let's add our kos-rs crate to gain access to current KallistiOS bindings. Open Cargo.toml in your text editor and add:

[dependencies]
kos = { package = "kos-rs", git = "https://github.com/dreamcast-rs/kos-rs" }

Now we can open up src/main.rs and write our "Hello, world!" example code:

fn main() {
    println!("Hello, world from Rust!");
}

Now we can use kos-cargo build to build our project. If all goes well, there will be a target/sh-elf/debug/hello.elf file that can be sent to the Dreamcast with dc-tool. If you have KOS_LOADER set in the environ.sh file, you can invoke it directly with kos-cargo run.

Creating a Rust project using kos-ports libraries

cube example in action

The gldc-cube example demonstrates creating a rotating 3D cube using Rust as the primary language, while calling C functions provided by the GLdc library available in kos-ports. This project's initial setup is done the same as the above hello example.

For this example, we'll need to convert JPEG textures to VQ-encoded textures at compile time using the vqenc utility includes with KallistiOS. To do this, we'll add a build.rs file to the root of the crate with the following code (abridged to show the conversion of only one texture):

fn main() {
    let kos_base = std::env::var("KOS_BASE").expect("Missing $KOS_BASE -- KallistiOS environment not sourced!");
    let vqenc_cmd = kos_base + "/utils/vqenc/vqenc";
    Command::new(&vqenc_cmd)
        .arg("-t")
        .arg("-v")
        .arg("rsrc/tex_claw.jpg")
        .output()
        .expect("vqenc on tex_claw.jpg failed!");
}

The texture files are then included in our project using the include_bytes! macro.

We will need to link the GLdc library in this example, but we don't need to do anything special to add the paths to this library, because adding the common paths to KallistiOS libraries is already done for us in the kos-rs crate. However, if in your project you need to include a separate unique library path, you can add it in your build.rs like so:

println!("cargo:rustc-link-search=native={}", lib_path);

The workings of this example's source code are too great to detail here line-by-line, but the example demonstrates how to use GLdc via the gldc-sys crate, which contains unsafe/raw bindings to GLdc. Check out this crate's source code for an example of declaring and binding external C functions, constants, and structures so they can be used in Rust code. Since the entirety of the gldc-cube example uses C FFI functions without safe wrappers, the main() source is wrapped in unsafe {}. In the future, this would be much less necessary as higher level safe Rust bindings to KallistiOS and other libraries become mature.

Creating a Rust library

Next, we'll demonstrate creating a Rust library with kos-cargo that can be included in other Dreamcast code. This will follow the addlib example. Once again, this project's initial setup is done the same as the above hello example, but you'll create the new project using kos-cargo new --lib addlib to specify that we're creating a library named addlib. You'll also need to add the following text to this project's Cargo.toml file:

[lib]
crate-type = ["staticlib"]

This tells Rust to build a static .a library archive file from our code, which is located in src/lib.rs:

#[no_mangle]
pub extern "C" fn print_added(a: isize, b: isize) {
    print!("{}", a + b);
}

#[no_mangle]
pub extern "C" fn add_integers(a: isize, b: isize) -> isize {
    a + b
}

The source code here starts similarly to the "Hello, world!" example, except we don't need to specify #![no_main] as this is a library which wouldn't have a main() function anyway.

Two simple functions are provided: one for adding two integers and returning the result, and another for adding two integers and printing the result as text. Because these functions use #[no_mangle] and are declared extern "C", they can be called by name in C code that links this library.

When built using kos-cargo build, a target/sh-elf/debug/libaddlib.a file will be generated. This can be linked into other projects to gain the use of these functions.

For example, this can be added to a standard Makefile-based KallistiOS project by editing the Makefile:

$(TARGET): $(OBJS)
	kos-cc -o $(TARGET) $(OBJS) -L/opt/toolchains/dc/rust/examples/cargo-addlib/target/sh-elf/debug -laddlib

Then, we can use the code in our C source:

/* Declare the external function from the Rust library */
int add_integers(int a, int b);

/* Use the function */
printf("Five plus six is %d\n", add_integers(5, 6));

Compiling individual modules into object files with rustc

If we'd like to mix C and Rust code in the same Makefile-based KallistiOS project without building an entirely separate library, we can do that as well. This is demonstrated in the hello-make example.

Instead of using kos-cargo, we can invoke the kos-rustc via a Makefile to build Rust modules. If we assume the Rust file is named example.rs, you'll need to add example.o as an object file in your Makefile's OBJS = declaration. For example, if the project has two source files hello_c.c and hello_rust.rs, our Makefile would have a line like this:

OBJS = hello_c.o hello_rust.o

The example code demonstrates starting a C main() function to call a Rust function which builds a String containing the "Hello, world!" text which is passed back to a C function which prints Strings.

Using the libc crate with KallistiOS

If any of the crates in your project's dependency tree requires the libc crate, you will need to override fetching the default crate with our custom libc crate with KallistiOS/Dreamcast definitions. This custom version was copied to /opt/toolchains/dc/rust/libc when building the Rust sysroot earlier. Add the following text to your Cargo.toml:

[patch.crates-io]
libc = { path = "/opt/toolchains/dc/rust/libc" }

Adjusting build flags

Build settings can be adjusted through the KOS_RCG_RUSTFLAGS variable in the environ.sh file under the ### Rust Flags section.

  • To pass arguments to the rustc frontend, e.g. to disable debug assertions:
    • export KOS_RCG_RUSTFLAGS="${KOS_RCG_RUSTFLAGS} -Cdebug-assertions=no"
  • To pass arguments to the GCC backend, e.g. to enable the -freorder-blocks-algorithm=simple optimization for Rust code:
    • export KOS_RCG_RUSTFLAGS="${KOS_RCG_RUSTFLAGS} -Cllvm-args=-freorder-blocks-algorithm=simple"