OCaml 5 Multicore Guide: Implementing Eio for High-Performance Concurrency

For over a decade, OCaml engineers have lived in a paradox: writing type-safe, high-performance systems code while restricted to a single core. The ecosystem relied on Lwt and Async—monadic concurrency libraries that simulated multitasking on a single OS thread. While effective for I/O-bound workloads, this approach introduced the "function coloring" problem (wrapping everything in promises) and hit a hard ceiling when CPU-bound tasks blocked the event loop.

OCaml 5 removes the Global Runtime Lock. With the introduction of Effect Handlers and Domains, we can now achieve direct-style concurrency (code looks synchronous but is asynchronous) and true parallelism.

This guide demonstrates how to architect a high-performance network service using Eio (Effects-based I/O), OCaml's modern concurrency library, bridging the gap between lightweight fibers and heavyweight domains.

The Architecture Shift: From Monads to Effects

To migrate effectively, you must understand the mechanical difference between legacy Lwt and modern Eio.

The Legacy Model (Lwt/Async)

In Lwt, concurrency is reified as a value: the Promise ('a Lwt.t). This is Cooperative Multithreading in user space.

Heap Allocation: Every asynchronous step allocates a promise node on the heap.
Callback Hell: Even with syntax sugar like let*, the underlying execution involves chaining closures.
The Blocker: If you calculate a SHA-256 hash in an Lwt callback, the entire event loop freezes. No other I/O happens until that calculation finishes.

The OCaml 5 Model (Eio)

OCaml 5 introduces Delimited Continuations via Effect Handlers.

Direct Style: Functions return 'a, not 'a Lwt.t. When an I/O operation (like read) blocks, the runtime captures the stack (the continuation) and suspends the fiber.
Scheduler Integration: The scheduler switches to another fiber. When the I/O completes, the original stack is resumed exactly where it left off.
Structured Concurrency: Unlike Lwt's detached promises, Eio enforces a strict hierarchy via Switches. A parent switch cannot exit until all child fibers complete, eliminating resource leaks.

The Implementation: A Multicore Job Processor

We will build a TCP server that accepts work items (integers) and performs a CPU-intensive calculation (Fibonacci) on them. Crucially, we will offload the CPU work to a separate Domain (OS Thread) so the accept loop (I/O) never blocks.

Prerequisites

Ensure you have an OCaml 5.0+ switch and eio_main installed.

opam install eio_main

The Code

open Eio.Std

(* 1. CPU-Intensive Workload
   This simulates a heavy computation that would normally block
   the Lwt event loop. *)
let rec heavy_computation n =
  if n <= 1 then n
  else heavy_computation (n - 1) + heavy_computation (n - 2)

(* 2. The Worker Logic
   This function runs inside a separate Domain. It receives work 
   from the main thread, computes it, and returns the result. 
   
   Note: We use Eio.Domain_manager to cross the domain boundary. *)
let run_worker domain_mgr n =
  Eio.Domain_manager.run domain_mgr (fun () ->
    traceln "[Domain %d] Processing req: %d" (Domain.self () :> int) n;
    heavy_computation n
  )

(* 3. Client Handler
   Handles a single TCP connection. It reads a line, offloads the work,
   and writes the response. *)
let handle_client domain_mgr flow addr =
  traceln "Accepted connection from %a" Eio.Net.Sockaddr.pp addr;
  
  (* Buffered reader for efficient line parsing *)
  let reader = Eio.Buf_read.of_flow ~max_size:1024 flow in
  
  try
    while true do
      let line = Eio.Buf_read.line reader in
      match int_of_string_opt line with
      | Some n ->
          (* CRITICAL: Offload CPU work to a parallel domain.
             The main fiber suspends here, but the event loop continues
             processing other connections. *)
          let result = run_worker domain_mgr n in
          
          let response = Printf.sprintf "Result: %d\n" result in
          Eio.Flow.copy_string response flow
      | None ->
          Eio.Flow.copy_string "Error: Invalid number\n" flow
    done
  with End_of_file ->
    traceln "Connection closed for %a" Eio.Net.Sockaddr.pp addr

(* 4. The Server Loop
   Sets up the listening socket and spawns a new fiber for each client. *)
let main ~net ~domain_mgr =
  (* A switch delimits the lifetime of our fibers. 
     When the switch finishes, all fibers are cancelled/awaited. *)
  Switch.run @@ fun sw ->
    let addr = `Tcp (Eio.Net.Ipaddr.V4.loopback, 8080) in
    let socket = Eio.Net.listen net ~sw ~backlog:128 ~reuse_addr:true addr in
    
    traceln "Server listening on port 8080...";
    
    (* Accept loop *)
    while true do
      Eio.Net.accept_fork ~sw socket ~on_error:(fun ex -> 
        traceln "Connection error: %a" Fmt.exn ex
      ) (handle_client domain_mgr)
    done

(* 5. Entry Point *)
let () =
  (* Eio_main.run initializes the scheduler and the environment *)
  Eio_main.run @@ fun env ->
    let net = Eio.Stdenv.net env in
    let domain_mgr = Eio.Stdenv.domain_mgr env in
    main ~net ~domain_mgr

Technical Breakdown

1. The Environment (`env`)

In OCaml 5 Eio, capabilities are explicit. You don't just "open a socket"; you request access to the Network capability from the environment (Eio.Stdenv.net env). This makes the code testable and capability-safe. Passing domain_mgr explicitly allows us to control exactly where parallel processing is permitted.

2. Structured Concurrency (`Switch.run`)

Look at the handle_client logic. In Lwt, if a client disconnected abruptly, you often had to manage promise cancellation manually or risk zombie processes. In Eio, Switch.run creates a scope. All fibers forked into this switch (via accept_fork) are tied to its lifecycle. If the switch is turned off (cancelled), all child fibers are automatically cancelled.

3. Crossing Boundaries (`Eio.Domain_manager`)

The most significant line is:

Eio.Domain_manager.run domain_mgr (fun () -> ... )

This is the bridge between Concurrency and Parallelism.

Concurrency (Fibers): Thousands of clients can connect. Eio schedules them on a single core using lightweight fibers. Context switching is nanosecond-scale.
Parallelism (Domains): When heavy_computation is needed, we don't want to block the fiber scheduler. Domain_manager.run spawns the task onto a pool of available OS threads (Domains).
Safety: OCaml 5's memory model allows these domains to share the heap safely. You don't need complex message passing for simple data; you can pass closures directly, provided you handle mutable state carefully (preferring immutability or Atomic types).

4. Buf_read and Flow

Direct-style I/O means Eio.Buf_read.line blocks the fiber, not the thread. Under the hood, Eio registers the file descriptor with io_uring (on Linux) or kqueue (on macOS) and suspends the effect. The scheduler immediately picks up the next fiber (e.g., another client connection).

Why This Matters

This architecture solves the fundamental scalability issue of OCaml backends.

Latency tail-end: In Lwt, a single request doing heavy JSON parsing or crypto could cause latency spikes for all concurrent requests. In this Eio model, CPU load is shunted to a different core, keeping the I/O loop responsive.
Code maintainability: The code reads top-to-bottom. There are no >>= operators, no Lwt.catch, and no need to lift pure functions into the monad. The stack trace you see in a debugger is the actual logical stack trace of your program.

By adopting Eio, you aren't just updating libraries; you are moving to a runtime model that aligns with modern hardware architecture.

Programming Tutorials

Search This Blog