Skip to main content

OCaml Multicore Guide: Eio vs Lwt for High-Performance Concurrency

 The release of OCaml 5.0 marked the end of the Global Interpreter Lock (GIL) era, introducing native support for shared-memory parallelism (Domains) and direct-style concurrency via Algebraic Effects.

However, this paradigm shift creates a friction point for production engineering. The OCaml ecosystem has spent over a decade building on monadic concurrency libraries, primarily Lwt and Async. These libraries rely on "function coloring"—where asynchronous functions return a specific wrapper type (e.g., 'a Lwt.t)—infecting the entire call stack.

Engineers must now decide: Do we continue paying the "monad tax" with Lwt, or do we migrate to Eio, the effects-based IO library that promises direct-style syntax, structured concurrency, and parallel scalability?

The Root Cause: Monadic Concurrency vs. Effect Handlers

To make the right architectural decision, you must understand how the runtime behaves in both scenarios.

1. The Lwt Model (Cooperative Monads)

Lwt simulates concurrency on a single core using an event loop.

  • Heap Allocation: Every asynchronous operation creates a Promise object on the heap.
  • Function Coloring: If fetch_data is async, process_data which calls it must also become async (returning 'a Lwt.t). This complicates refactoring and higher-order functions.
  • Single Domain: Standard Lwt runs on a single OS thread. It cannot utilize OCaml 5's multicore capabilities for CPU-bound tasks without forking processes or using the Lwt_preemptive detach mechanism (which has overhead).

2. The Eio Model (Direct-Style Effects)

Eio leverages OCaml 5 Effect Handlers. When an I/O operation blocks, the runtime captures the current stack (the continuation) and suspends the fiber.

  • Stack vs. Heap: Continuations are lighter than promise chains.
  • No Coloring: Asynchronous code looks synchronous. Eio.Net.accept returns a socket, not a promise of a socket. You can use standard standard library functions (like List.map) without needing monadic equivalents (like Lwt_list.map_p).
  • Structured Concurrency: Eio enforces resource safety via Switches. If a switch context closes (or fails), all fibers spawned within it are cancelled automatically.
  • Parallelism: Eio can dispatch work across multiple Domains (OS threads), allowing true parallelism for CPU-heavy request processing.

The Solution: Migrating from Monads to Effects

Below is a comparative implementation of a high-performance TCP Echo Server. We will demonstrate the syntax gap, the resource management differences, and a strategy for interoperability.

Prerequisite

Ensure you are running OCaml 5.x and have the required packages:

opam install eio_main lwt lwt_ppx

1. The Legacy Approach: Lwt

In Lwt, we manage the event loop manually and chain promises using the let* syntax. Note that if the client disconnects abruptly, we must manually ensure exception handling prevents the server from crashing.

(* echo_lwt.ml *)
open Lwt.Infix

let handle_connection input_channel output_channel =
  let rec loop () =
    (* Reads allow context switching via the Lwt engine *)
    Lwt_io.read_line_opt input_channel >>= function
    | Some msg ->
        Lwt_io.write_line output_channel msg >>= loop
    | None -> Lwt.return_unit
  in
  (* Catch errors to prevent bringing down the main loop *)
  Lwt.catch loop (fun exn ->
      Lwt_io.printf "Error: %s\n" (Printexc.to_string exn))

let accept_connection _conn_addr fd =
  let input_channel = Lwt_io.of_fd ~mode:Lwt_io.Input fd in
  let output_channel = Lwt_io.of_fd ~mode:Lwt_io.Output fd in
  (* Detach connection handling so accept loop continues immediately *)
  Lwt.async (fun () ->
      handle_connection input_channel output_channel >>= fun () ->
      Lwt_io.close input_channel)

let create_server port =
  let open Lwt_unix in
  let sockaddr = ADDR_INET (Unix.inet_addr_any, port) in
  let sock = socket PF_INET SOCK_STREAM 0 in
  setsockopt sock SO_REUSEADDR true;
  bind sock sockaddr;
  listen sock 10; (* Backlog *)
  
  Printf.printf "Lwt Server listening on port %d\n%!" port;
  
  let rec loop () =
    accept sock >>= fun (fd, addr) ->
    accept_connection addr fd;
    loop ()
  in
  loop ()

let () =
  Lwt_main.run (create_server 8080)

2. The Modern Approach: Eio

Eio code reads top-to-bottom. We use Eio.Switch to bound the lifetime of resources. We also use Eio.Buf_read for high-performance buffered parsing.

Crucially, this example uses Eio.Domain_manager to offload the connection handling to other cores if necessary, though for simple IO, a single domain with fibers is sufficient.

(* echo_eio.ml *)
open Eio.Std

let handle_client flow _addr =
  (* Buffered reader for efficiency *)
  let reader = Eio.Buf_read.of_flow flow ~max_size:4096 in
  try
    while true do
      let line = Eio.Buf_read.line reader in
      Eio.Flow.copy_string (line ^ "\n") flow
    done
  with End_of_file -> ()

let main ~net ~domain_mgr =
  (* Structured Concurrency: The switch defines the scope of resources *)
  Eio.Switch.run @@ fun sw ->
  let addr = `Tcp (Eio.Net.Ipaddr.V4.any, 8081) in
  let socket = Eio.Net.listen net ~sw ~backlog:10 ~reuse_addr:true addr in
  
  Eio.traceln "Eio Server listening on port 8081";

  while true do
    Eio.Net.accept_fork ~sw socket ~on_error:(fun exn -> 
      Eio.traceln "Connection error: %a" Fmt.exn exn
    ) (fun flow addr ->
       (* Option A: Handle on same domain (Green threads) *)
       handle_client flow addr
       
       (* Option B: Offload to another core for CPU heavy work: 
       Eio.Domain_manager.run domain_mgr (fun () -> handle_client flow addr) 
       *)
    )
  done

let () =
  (* Auto-detects backend: io_uring (Linux), kqueue (macOS), or luv *)
  Eio_main.run @@ fun env ->
  main ~net:(Eio.Stdenv.net env) ~domain_mgr:(Eio.Stdenv.domain_mgr env)

3. The Migration Strategy: Lwt_eio

You cannot rewrite a million lines of Lwt code overnight. The bridge Lwt_eio allows you to run an Lwt event loop inside an Eio fiber. This lets you write new backend logic in Eio while maintaining legacy Lwt libraries (like cohttp-lwt or database drivers).

(* interop.ml *)
open Eio.Std

(* A legacy Lwt function *)
let legacy_sleep_and_print () =
  let open Lwt.Syntax in
  let* () = Lwt_unix.sleep 0.1 in
  Lwt_io.printl "Hello from Legacy Lwt!"

let () =
  Eio_main.run @@ fun env ->
  Eio.Switch.run @@ fun sw ->
  
  (* Start the Lwt engine interoperability layer *)
  Lwt_eio.with_event_loop ~clock:env#clock @@ fun () ->
    
  (* Run Lwt promise within Eio, blocking the fiber but not the domain *)
  Lwt_eio.run_lwt (fun () -> legacy_sleep_and_print ());
  
  Eio.traceln "Back to Eio direct style!"

Why This Implementation Works

Structured Concurrency via Eio.Switch

In the Lwt example, Lwt.async creates a "detached" promise. If the main server loop crashes, those detached promises might keep running or leak file descriptors until the OS cleans them up. In the Eio example, Eio.Net.accept_fork attaches the new connection fiber to the provided switch ~sw. If the switch goes out of scope (or an exception bubbles up to it), Eio cancels all child fibers and closes their resources (sockets) deterministically.

Backends matter: io_uring

Eio_main.run automatically selects the best backend. On Linux, it uses io_uring. Lwt typically defaults to epoll. io_uring reduces system call overhead significantly by submitting batch IO requests via a ring buffer shared between kernel and user space. This makes Eio inherently faster for I/O heavy workloads on modern Linux kernels.

The Stack Trace Benefit

When an Lwt program crashes, the stack trace is often unintelligible because it points to the event loop internals rather than your logic. Because Eio runs on the direct stack (interrupted by effects), stack traces remain readable and point directly to the line of code in your logic that failed.

Conclusion

Migrating to Eio is not just a syntax update; it is an architectural upgrade to OCaml 5's multicore runtime.

  1. New Projects: Use Eio. The cognitive load is lower (no monads), and performance ceilings are higher (multicore).
  2. Existing Projects: Use Lwt_eio. Keep your existing HTTP handlers or database wrappers in Lwt, but wrap the entry point in Eio to start utilizing Domains for background processing or heavy computation.
  3. Performance: For I/O bound work, Eio with io_uring outperforms Lwt. For CPU bound work, Eio + Domains is the only viable path in OCaml.
Labels: