►A Racket client for the Maelstrom distributed systems test harness
A Racket client for the Maelstrom distributed systems test harness
1 Introduction
2 Creating a node
node?
make-node
3 Adding handlers
add-handler
4 Running a node
run
5 Sending messages
send
respond
rpc
6 Other node operations
node-id
known-peers
7 The Message API
message?
message-ref
message-body
message-sender
message-id
message-type
make-message
8 Access to key-value stores
lin-kv
lww-kv
seq-kv
kv-read
kv-write
kv-cas
9 Logging
8.12
  top  contents  ← prev  up  next → 

A Racket client for the Maelstrom distributed systems test harness🔗ℹ

Nikhil Marathe <racket-packages@me.nikhilism.com>

 (require maelstrom) package: maelstrom

Maelstrom is a workbench for learning distributed systems by writing your own. This is a library to allow implementing a Maelstrom server node in Racket, similar to existing implementations in other languages.

Familiarity with Maelstrom terminology is assumed. The following resources are handy:

1 Introduction🔗ℹ

The typical implementation of a node will involve:

  1. Creating a node

  2. Adding handler functions

  3. In a main submodule, running the node.

For example, Challenge #1 Echo would be written as:

(require maelstrom
         maelstrom/message)
 
(define node (make-node))
(add-handler node
             "echo"
             (lambda (req)
               (respond req (hash 'echo (message-ref req 'echo)))))
 
(module+ main
  (run node))

built into a binary using:

raco exe echo.rkt

and then run as:

maelstrom test -w echo --bin echo --node-count 1 --time-limit 10

2 Creating a node🔗ℹ

procedure

(node? n) → boolean?

  n : any/c

procedure

(make-node) → node?

Creates a new node.

3 Adding handlers🔗ℹ

procedure

(add-handler node type proc) → void

  node : node?
  type : string?
  proc : (message? . -> . any/c)
Add a handler for messages received with the 'type key in the message body set to this string. Only one handler can exist for a particular type. Any handler added for "init" will get replaced by the default initialization handler!

The handler receives a message (see the The Message API). Its return value is ignored. Each handler invocation runs in its own thread to not block other handlers. Errors are logged to the maelstrom logger.

4 Running a node🔗ℹ

procedure

(run node) → void

  node : node?
Begins running a node. This is a blocking call. It will use current-input-port to read messages sent to the node and current-output-port to write outgoing messages. To ensure preconditions are satisfied, run expects a "init" message as the first message.

run will only return once current-input-port is closed and all handlers have returned. To forcibly shut it down, you can spawn it in a separate thread and use kill-thread.

It is undefined behavior to call run on the same node more than once!

5 Sending messages🔗ℹ

This library uses Racket’s parameters and dynamic scoping to make responding to messages more convenient. The handler code does not need to refer to the node?. Instead handlers are invoked with the dynamic context suitably modified such that all these functions no which node to act on.

Any sub-threads spawned by handlers will inherit the correct bindings automatically. See this solution to challenge #3d that spawns a spawn-minder thread within the topology handler, and the spawn-minder thread is still able to use rpc.

procedure

(send dest msg) → void

  dest : string?
  msg : hash?
Send a new message to the peer with id dest. msg should contain at least a 'body. It is recommended to use make-message. No response is expected from the peer. To listen for responses, use rpc.

procedure

(respond request [additional-body]) → void

  request : message?
  additional-body : hash? = (hash)
A convenience function to use to respond to the message which caused this handler to be run. This will automatically fill in the appropriate 'type and 'in_reply_to, and send the message to the original sender of request.

request will usually be the message received by the handler procedure. Any additional body parameters can be supplied in the additional-body hash. See the example in Introduction.

procedure

(rpc dest msg proc) → void

  dest : string?
  msg : hash?
  proc : (message? . -> . any/c)
Sends msg to dest similar to send. In addition, proc will be called when the peer responds to this specific msg. This is correlated by 'msg_id and 'in_reply_to according to the Maelstrom protocol. proc receives the message sent by the peer.

Similar to add-handler handlers, proc is called in a new thread.

Note that proc is stored until a response is received. This means if peers never respond to messages, memory leaks are possible. There is no solution to this right now, as Maelstrom is not for building production services.

6 Other node operations🔗ℹ

procedure

(node-id n) → (or/c any/c #f)

  n : node?
Returns the id assigned to this node. Returns #f until the node is initialized. This is guaranteed to be set when called within message handlers.

procedure

(known-peers n) → (listof string?)

  n : node?
A list of peers of this node as notified by the "init" message’s "node_ids" field. The current node’s id is not present in this list. To receive the topology sent by most Maelstrom workloads, a custom handler for "topology" should be added.

7 The Message API🔗ℹ

 (require maelstrom/message) package: maelstrom

Messages are simply jsexpr?s with some additional semantics required by the Maelstrom protocol. Because of this, all keys are always Racket symbols.

procedure

(message? msg) → bool?

  msg : any/c
Returns true if msg is a valid Maelstrom protocol message.

procedure

(message-ref msg key) → jsexpr?

  msg : message?
  key : symbol?
Helper to directly access key in the msg’s 'body.

procedure

(message-body msg) → jsexpr?

  msg : message?
Get the entire msg 'body.

procedure

(message-sender msg) → jsexpr?

  msg : message?
Get the id of the node that send msg.

procedure

(message-id msg) → jsexpr?

  msg : message?
Get the id of msg.

procedure

(message-type msg) → jsexpr?

  msg : message?
Get the 'type of the msg.

procedure

(make-message body) → (hash/c symbol? jsexpr?)

  body : jsexpr?
Create a partially valid message-like hash based where the body is inserted as the 'body in the resulting hash. This can be used to create messages that can be sent to send or rpc.

(add-handler
 node
 "my-message"
 (lambda (req)
   (send "n3"
         (make-message
          (hash 'type "another-type"
                'field1 58
                'field2 (list "multiple" "items"))))))

8 Access to key-value stores🔗ℹ

 (require maelstrom/kv) package: maelstrom

Maelstrom provides a few different key-value (KV) stores that your nodes can use. These are exposed via the kv module.

(require maelstrom/kv)
 
; Read my-key from the sequentially consistent store.
; Return 0 if the key does not exist in the store.
(kv-read seq-kv "my-key" 0)

value

lin-kv : kv?

The linearizable key-value store.

value

lww-kv : kv?

The last-write-wins key-value store.

value

seq-kv : kv?

The sequentially consistent key-value store.

procedure

(kv-read kv k default) → jsexpr?

  kv : kv?
  k : string?
  default : jsexpr?
Read key k from the kv store. If k is not present, default is returned.

procedure

(kv-write kv k v) → void

  kv : kv?
  k : string?
  v : jsexpr?
Write value v to key k.

procedure

(kv-cas kv    
  k    
  from    
  to    
  [#:create-if-missing? create-if-missing?]) → bool?
  kv : kv?
  k : string?
  from : jsexpr?
  to : jsexpr?
  create-if-missing? : bool? = #f
Attempts a compare-and-swap on key k. This operation will succeed (and return #t) if k had the value from, and will atomically set k to to. If k is not from then the operation will fail, returning #f.

If create-if-missing? is #t, then the key is created if it does not exist. If create-if-missing? is #f, then an error will be raised.

9 Logging🔗ℹ

The library logs to the maelstrom logger. As an example, to see debug messages, you can run the program with the PLTSTDERR="debug@maelstrom" environment variable.

  top  contents  ← prev  up  next →