Volume R · The TensorCore.si Library

Rust

Fast as C++, safe by design.

Rust catches memory bugs and data races at compile time, without a garbage collector. It is increasingly used for AI tooling and inference servers, and Hugging Face, Mozilla, Microsoft and Linux all ship Rust code.

Prologue

Why learn Rust?

Rust asks more of you at first, mostly through its ownership rules, but it rewards you with programs that are fast, use little memory and simply do not have whole classes of bugs. Many developers say it changed how they write code in every language.

Memory safety without a garbage collector: no null-pointer crashes, use-after-free or data races in safe code.
C-level speed: ideal for inference servers, tokenizers and data pipelines.
Great tooling: Cargo builds, tests, formats and publishes with one command.
AI ecosystem growing fast: Hugging Face's tokenizers and Candle, the Polars data library and many inference servers are Rust.

Prologue

Set up and run your first program

Install Rust with rustup from rust-lang.org. It gives you the compiler (rustc) and Cargo, the build tool and package manager.

terminal

rustc --version
cargo new hello_ai
cd hello_ai
cargo run

src/main.rs

fn main() {
    let name = "Ada";
    println!("Hello, AI! Welcome to Rust, {name}.");
}

Tipcargo run compiles and runs in one step. Use cargo run --release when measuring speed; debug builds are much slower.

Chapter 1

Variables and types

Variables are immutable by default: add mut to change them. Rust infers types but you can write them. Common types: i32, i64, u64, f32, f64, bool, char, &str (borrowed text) and String (owned, growable text).

You can shadow a variable by declaring it again with let, even with a new type.

variables.rs

fn main() {
    let layers = 32;                 // i32, immutable
    let mut tokens: u32 = 512;       // mutable
    tokens += 128;
    let lr: f64 = 3e-4;
    let model: &str = "llama3.1";    // string slice
    let mut owned = String::from("Llama");
    owned.push_str(" 3.1");

    let size = "8";                  // shadowing
    let size: u32 = size.parse().unwrap();

    println!("{model}: {layers} layers, {tokens} tokens, lr {lr}, {owned}, {size}B");
}

Chapter 2

Control flow and match

if is an expression, so it can return a value. loop repeats forever (until break), while repeats on a condition and for iterates over ranges and collections. match compares against patterns and must cover every case.

control.rs

fn main() {
    let vram = 24;
    let tier = if vram >= 80 { "data center" } else { "desktop" };
    println!("{tier}");

    for i in 0..3 {
        println!("epoch {i}");
    }

    let precision = 8;
    let name = match precision {
        32 => "FP32",
        16 => "FP16",
        8 => "FP8",
        4 | 6 => "FP4/FP6",
        _ => "other",
    };
    println!("{name}");
}

Chapter 3

Functions and closures

Functions declare parameter and return types. The last expression without a semicolon is the return value. Closures (|x| x * 2) capture variables from their surroundings and work with iterators.

functions.rs

fn memory_gb(params_b: f64, bytes: f64) -> f64 {
    params_b * bytes          // no semicolon: this is returned
}

fn main() {
    println!("{} GB", memory_gb(70.0, 2.0));
    let factor = 3;
    let triple = |x: i32| x * factor;     // closure captures factor
    println!("{}", triple(14));
}

Chapter 4

Ownership and borrowing

This is the heart of Rust. Every value has exactly one owner; when the owner goes out of scope, the value is freed. Assigning a String to another variable moves it.

To use a value without taking it, borrow it with & (read-only, many allowed) or &mut (writable, only one at a time). The compiler checks these rules, which is how Rust prevents memory bugs and data races.

ownership.rs

fn length(s: &String) -> usize {   // borrows, does not take ownership
    s.len()
}

fn add_suffix(s: &mut String) {    // mutable borrow
    s.push_str("-instruct");
}

fn main() {
    let a = String::from("llama");
    let b = a;                        // ownership moves to b
    // println!("{a}");               // error: a was moved

    let mut name = b.clone();         // explicit deep copy
    add_suffix(&mut name);
    println!("{} has {} chars", name, length(&name));
}

Key ideaIf the borrow checker rejects your code, it is usually pointing at a real problem, such as two parts of the program changing the same data at once. Read the error message: Rust's are famously helpful.

Chapter 5

Vectors, hash maps and iterators

Vec<T> is a growable array and HashMap<K, V> a dictionary. Iterators chain map, filter and sum and compile to code as fast as a hand-written loop.

collections.rs

use std::collections::HashMap;

fn main() {
    let mut losses = vec![0.9, 0.4, 0.7, 0.2];
    losses.push(0.1);
    let mean: f64 = losses.iter().sum::<f64>() / losses.len() as f64;
    let low: Vec<f64> = losses.iter().copied().filter(|&l| l < 0.5).collect();
    println!("mean {mean:.2}, low {low:?}");

    let mut vram: HashMap<&str, u32> = HashMap::new();
    vram.insert("H100", 80);
    vram.insert("B200", 180);
    for (gpu, gb) in &vram {
        println!("{gpu}: {gb} GB");
    }
}

Chapter 6

Structs, enums and traits

A struct groups data and impl adds methods. An enum can hold different kinds of data in each variant, which makes it perfect for states and messages. A trait describes shared behaviour, like an interface. #[derive] generates common traits automatically.

structs.rs

#[derive(Debug, Clone)]
struct Gpu {
    name: String,
    vram_gb: u32,
}

impl Gpu {
    fn fits(&self, model_gb: u32) -> bool {
        model_gb <= self.vram_gb
    }
}

enum Message {
    User(String),
    Tool { name: String, result: String },
}

trait Describe {
    fn describe(&self) -> String;
}

impl Describe for Gpu {
    fn describe(&self) -> String {
        format!("{} with {} GB", self.name, self.vram_gb)
    }
}

fn main() {
    let g = Gpu { name: "RTX 5090".into(), vram_gb: 32 };
    println!("{} fits 16 GB: {}", g.describe(), g.fits(16));
    let msgs = vec![Message::User("hi".into()), Message::Tool { name: "search".into(), result: "ok".into() }];
    for m in msgs {
        match m {
            Message::User(t) => println!("user: {t}"),
            Message::Tool { name, result } => println!("{name} -> {result}"),
        }
    }
}

Chapter 7

Errors with Result and Option

Rust has no exceptions and no null. A function that can fail returns Result<T, E> (Ok or Err); a value that may be missing is an Option<T> (Some or None). The ? operator returns the error early, keeping code short.

errors.rs

use std::fs;
use std::num::ParseIntError;

fn parse_layers(s: &str) -> Result<u32, ParseIntError> {
    let n: u32 = s.trim().parse()?;   // returns Err early if parsing fails
    Ok(n)
}

fn main() {
    match parse_layers("32") {
        Ok(n) => println!("{n} layers"),
        Err(e) => println!("bad input: {e}"),
    }
    let config = fs::read_to_string("config.txt").unwrap_or_else(|_| "default".to_string());
    let first = vec![1, 2, 3].first().copied();   // Option<i32>
    println!("{config} {first:?}");
}

Chapter 8

Modules, crates and Cargo

Code is organised in modules (mod) inside a crate (a library or program). Dependencies are crates from crates.io, listed in Cargo.toml and downloaded by Cargo.

Cargo.toml

[package]
name = "hello_ai"
version = "0.1.0"
edition = "2021"

[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
reqwest = { version = "0.12", features = ["blocking", "json"] }

Chapter 9

Files and JSON with serde

std::fs reads and writes files. The serde crate converts Rust structs to and from JSON with one derive line, which is ideal for model configs and API messages.

files.rs

use serde::{Deserialize, Serialize};
use std::fs;

#[derive(Serialize, Deserialize, Debug)]
struct Results {
    model: String,
    accuracy: f64,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let r = Results { model: "llama3.1".into(), accuracy: 0.91 };
    fs::write("results.json", serde_json::to_string_pretty(&r)?)?;
    let back: Results = serde_json::from_str(&fs::read_to_string("results.json")?)?;
    println!("{back:?}");
    Ok(())
}

Chapter 10

Fearless concurrency

Threads share data safely through Arc (shared ownership) and Mutex (one writer at a time), or communicate through channels. The compiler refuses code that could cause a data race. For network-heavy programs, the tokio runtime provides async/await.

threads.rs

use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
    let total = Arc::new(Mutex::new(0u64));
    let mut handles = vec![];
    for id in 0..4 {
        let total = Arc::clone(&total);
        handles.push(thread::spawn(move || {
            let work: u64 = (0..1_000_000u64).sum();
            *total.lock().unwrap() += work;
            println!("thread {id} done");
        }));
    }
    for h in handles {
        h.join().unwrap();
    }
    println!("total = {}", total.lock().unwrap());
}

Chapter 11

Testing and tooling

Tests live next to your code in a #[cfg(test)] module and run with cargo test. cargo fmt formats, cargo clippy gives hundreds of helpful lints and cargo doc builds documentation.

lib.rs

pub fn memory_gb(params_b: f64, bytes: f64) -> f64 {
    params_b * bytes
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn fp16_memory() {
        assert_eq!(memory_gb(8.0, 2.0), 16.0);
    }
}

Part II · AI with Rust

AI with Rust

Rust is used where AI needs to be fast, small and reliable: tokenizers, inference servers, data processing and running models on edge devices or in the browser through WebAssembly. Its ML frameworks are younger than Python's but growing quickly.

The essential libraries

CandleHugging Face's minimalist ML framework with CPU, CUDA and Metal.

BurnA full deep-learning framework with training and many back ends.

tch-rsRust bindings to LibTorch (PyTorch's C++ core).

ortRust wrapper for ONNX Runtime inference.

tokenizersHugging Face's fast tokenizers, written in Rust.

mistral.rsFast LLM inference server.

PolarsDataFrames that are often many times faster than pandas.

linfaClassic machine learning, like scikit-learn.

Matrix multiplication on the GPU with Candle

Candle runs the same code on CPU or an NVIDIA GPU. Matrix multiplication is the operation Tensor Cores accelerate.

main.rs

// Cargo.toml: candle-core = "0.8"   (add features = ["cuda"] for NVIDIA GPUs)
use candle_core::{Device, Tensor};

fn main() -> candle_core::Result<()> {
    let device = Device::cuda_if_available(0)?;
    let a = Tensor::randn(0f32, 1., (2, 3), &device)?;
    let b = Tensor::randn(0f32, 1., (3, 4), &device)?;
    let c = a.matmul(&b)?;
    println!("{c}");
    Ok(())
}

Check crates.io for the latest Candle version. Its repository includes ready-to-run examples for Llama, Whisper and Stable Diffusion.

Part III · Build an AI agent

Build an AI agent in Rust

This agent uses reqwest for HTTP and serde_json for messages. Rust's enums and pattern matching make the "did the model call a tool?" logic clear and safe.

Your goal→Model plans→Tool call→Your code runs the tool→Result back to model→Final answer

src/main.rs

// Cargo.toml: reqwest = { version = "0.12", features = ["blocking", "json"] }, serde_json = "1"
use serde_json::{json, Value};

fn get_weather(city: &str) -> String {
    format!("It is 24°C and sunny in {city}.")
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::blocking::Client::new();
    let tools = json!([{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    }]);
    let mut messages = vec![json!({"role": "user", "content": "Should I take an umbrella in Lahore today?"})];

    loop {
        let reply: Value = client
            .post("http://localhost:11434/v1/chat/completions")
            .json(&json!({"model": "llama3.1", "messages": messages, "tools": tools}))
            .send()?
            .json()?;
        let msg = reply["choices"][0]["message"].clone();
        messages.push(msg.clone());

        let calls = match msg["tool_calls"].as_array() {
            Some(c) if !c.is_empty() => c.clone(),
            _ => {
                println!("{}", msg["content"].as_str().unwrap_or(""));
                break;
            }
        };
        for call in calls {
            let args: Value = serde_json::from_str(call["function"]["arguments"].as_str().unwrap_or("{}"))?;
            let result = get_weather(args["city"].as_str().unwrap_or(""));
            messages.push(json!({"role": "tool", "tool_call_id": call["id"], "content": result}));
        }
    }
    Ok(())
}

For production agents, switch to async reqwest with tokio so many conversations run at once, and look at crates such as async-openai and rig.

Run it locallyInstall Ollama, run ollama pull llama3.1, and keep it running. The code talks to its OpenAI-compatible API at http://localhost:11434/v1, so the same program also works with vLLM, llama.cpp server or a cloud API by changing the URL, model name and adding an API key.

Part IV · Your path

Projects to build

CLI to-do appA command-line task manager saving to a JSON file.

Word counterCount words in large files fast, then parallelise it.

TokenizerImplement byte-pair encoding, the tokenizer used by LLMs.

Web APIA REST API with axum that serves model predictions.

Local LLM chatRun a quantized Llama model with Candle or mistral.rs.

Browser demoCompile Rust to WebAssembly and run inference in a web page.

Reference

Rust cheat sheet

Task	Syntax
Print	`println!("{x} and {y:?}");`
Variable	`let x = 5; let mut y = 6;`
String	`let s = String::from("hi");`
Vector	`let mut v = vec![1, 2]; v.push(3);`
Function	`fn add(a: i32, b: i32) -> i32 { a + b }`
Match	`match x { 1 => .., _ => .. }`
Borrow	`fn f(s: &str) {} f(&owned);`
Struct	`struct P { x: f64 }`
Error early	`let n: u32 = s.parse()?;`
Iterator	`v.iter().map(\|x\| x * 2).collect::<Vec<_>>()`
Run fast	`cargo run --release`

Reference

Common mistakes

Fighting the borrow checker with clone() everywhere: fine while learning, but restructure code so data has a clear owner.
unwrap() in production: it panics on errors. Use ? and handle errors properly.
Confusing String and &str: functions should usually take &str and return String.
Benchmarking debug builds: always use --release.
Blocking inside async code: use async versions of I/O within tokio.

Reference

Where to go next

The Rust Programming Language ("the book") — free, the standard way to learn
Rust by Example
Rustlings exercises
Candle

On TensorCore.si: LLM inference simulator, LLM quantization explained.

Keyboard: [ previous language · ] next language · / search the library

Rust

Why learn Rust?

Set up and run your first program

Variables and types

Control flow and match

Functions and closures

Ownership and borrowing

Vectors, hash maps and iterators

Structs, enums and traits

Errors with Result and Option

Modules, crates and Cargo

Files and JSON with serde

Fearless concurrency

Testing and tooling

AI with Rust

The essential libraries

Matrix multiplication on the GPU with Candle

Build an AI agent in Rust

Rust learning roadmap

Beginner

Intermediate

Advanced

Projects to build

Rust cheat sheet

Common mistakes

Where to go next