14. Packages

Goal

Build a service on top of the rumor package, and use somebody else's package next to your own.

New in this chapter

use … as base form use-as spec/06 §6.3

Steps

Until now each package stood alone. Real systems are built in layers: a service uses a gossip library, an application uses the service. In Synod a package uses another one with use, which you know, and running is transitive: a node of a package also runs every package that package uses. A service.Node is also a rumor.Node.

Step 1 adds a service package that uses rumor. The service answers a question with what its rumor package knows. A client node is sent an Ask that names a server; it sends a Question; the server answers. The line that matters is n.rumor.Known: from outside, a field is reached through the package that declares it, node.package.Field, and it works because the node runs rumor. The demo builds a cluster of service nodes and hands the list to rumor.Connect, which wants list of rumor.Node: a service.Node is accepted wherever a rumor.Node is expected, because running service runs rumor:

Step 1

# package main
use example.com/tutorial/rumor
use example.com/tutorial/service
use std/sim

define main
  set nodes to (sim.Cluster of 5 running service)
  rumor.Connect nodes
  deliver (message rumor.Rumor with Text "The leader is n1") to (item 0 of nodes)
  after 3s
    deliver (message service.Ask with Of (item 3 of nodes)) to (item 4 of nodes)

# package service
use example.com/tutorial/rumor
use std/net
use std/sim

Ask has
  set Of to sim.Nobody

Question has
  set Topic to ""

Answer has
  set Text to ""

Node has
  set Heard to ""

# A client is told which server to ask
when [n: Node] (sim.Delivered Ask)
  net.Send (message Question with Topic "rumor") from n to Ask.Of

# A server answers with what its rumor package knows
when [n: Node] (net.Received Question from client)
  net.Send (message Answer with Text n.rumor.Known) from n to client

when [n: Node] (net.Received Answer from server)
  set n.Heard to Answer.Text

The invariants of chapter 11 sit in rumor and make it harder to reuse. A rule belongs to whoever wants it, so step 2 moves them to a package of their own, checks. It declares no node, no start and no message: only phrases and always. A package with an invariant is active as soon as a program imports it, even if the program never names it, which is why the demos import checks and nothing else from it. The rules read n.rumor.Known because they look at rumor nodes:

Step 2

# package checks
use example.com/tutorial/rumor
use std/lists
use std/sim

# True while the running nodes know at most one rumor between them
define Agree
  set texts to {}
  each n in (sim.NodesRunning rumor)
    if ((n.rumor.Known is not "") and (n.rumor.Known is not in texts))
      set texts to (lists.Append n.rumor.Known to texts)
  result ((lists.Size texts) at most 1)

# Must hold after every step of the run
always Agree

# How many nodes are down
define Down
  set count to 0
  each n in sim.Nodes
    if (not (sim.IsUp n))
      set count to (count plus 1)
  result count

# Fewer than three nodes down, always
always (Down less than 3)

# package main
use example.com/tutorial/checks
use example.com/tutorial/rumor
use example.com/tutorial/rumorlook
use std/lesson
use std/net
use std/sim

define main
  sim.Seed 4
  set nodes to (sim.Cluster of 6 running rumor)
  set left to {(item 0 of nodes), (item 1 of nodes), (item 2 of nodes)}
  set right to {(item 3 of nodes), (item 4 of nodes), (item 5 of nodes)}
  rumor.Connect nodes
  lesson.Note "n1 hears a rumor and starts telling it."
  deliver (message rumor.Rumor with Text "The leader is n1") to (item 0 of nodes)
  net.DropNext "Rumor" from (item 0 of nodes) to (item 1 of nodes)
  after 300ms
    lesson.Note "The network splits in two."
    net.Partition left from right
  after 3s
    lesson.Note "The network heals. The rumor can cross again."
    net.Heal

# package rumor
use std/disk
use std/lists
use std/net
use std/random
use std/sim

Rumor has
  set Text to ""

Ack has
  set Text to ""

Learned has
  set Text to ""

Join has
  set Peers to {}

Node has
  set Peers to {}
  set Known to ""
  set Acked to {}
  set GaveUp to 0

# Every node in the list is told about all the others
define Connect [ns: list of Node]
  each n in ns
    deliver (message Join with Peers (lists.Without n from ns)) to n

# The peers that have not confirmed the rumor yet
define [n: Node] Pending
  set rest to {}
  each p in n.Peers
    if (p is not in n.Acked)
      set rest to (lists.Append p to rest)
  result rest

# Run b now. If p has not confirmed 300ms later, run it again, up to k times in all, then run c
define [n: Node] Retry [k: number] times for [p: node] [b: block] otherwise [c: block]
  b
  after 300ms
    if (p is not in n.Acked)
      if (k more than 1)
        n.Retry (k minus 1) times for p
          b
        otherwise
          c
      else
        c

# Tell one random peer that has not confirmed, and again until all have
define [n: Node] Tell
  set open to n.Pending
  if ((lists.Size open) more than 0)
    set peer to (random.Pick open)
    n.Retry 3 times for peer
      net.Send (message Rumor with Text n.Known) from n to peer
    otherwise
      set n.GaveUp to (n.GaveUp plus 1)
    after (random.TimeBetween 200ms and 600ms)
      n.Tell

# Believe a rumor the first time it is heard, keep it on disk, announce it, and start telling it
define [n: Node] Learn [t: text]
  if (n.Known is "")
    set n.Known to t
    disk.Write t at "known" on n
    emit (message Learned with Text t) by n
    n.Tell

# After a restart the fields are fresh, but the disk remembers
define [n: Node] start
  set n.Peers to (disk.Read "peers" on n or {})
  set n.Known to (disk.Read "known" on n or "")
  if (n.Known is not "")
    n.Tell

# Peers are kept on disk too
when [n: Node] (sim.Delivered Join)
  set n.Peers to Join.Peers
  disk.Write n.Peers at "peers" on n

when [n: Node] (sim.Delivered Rumor)
  n.Learn Rumor.Text

# Confirm every telling, even of a rumor already known
when [n: Node] (net.Received Rumor from sender)
  n.Learn Rumor.Text
  net.Send (message Ack with Text Rumor.Text) from n to sender

when [n: Node] (net.Received Ack from sender)
  if (sender is not in n.Acked)
    set n.Acked to (lists.Append sender to n.Acked)

# package main
use example.com/tutorial/checks
use example.com/tutorial/rumor
use example.com/tutorial/service
use std/sim

define main
  set nodes to (sim.Cluster of 5 running service)
  rumor.Connect nodes
  deliver (message rumor.Rumor with Text "The leader is n1") to (item 0 of nodes)
  after 3s
    deliver (message service.Ask with Of (item 3 of nodes)) to (item 4 of nodes)

The next package is not yours. Packages come from Git repositories, and a module says which ones it needs in synod.mod. We want the gossip package of the examples repository, tagged v0.2.1. Step 3 adds the line

require github.com/synodlang/examples v0.2.1

and synod get downloads it into vendor/ (git ignores that folder) and writes a file synod.sum that pins the exact commit, so the program replays the same years from now. Commit synod.sum, never vendor/. Everything works offline once fetched.

We also write our own package called gossip, and it builds on theirs. Both are named gossip, so the import of the other would be ambiguous inside ours. That is the one case for use … as: the import is given another name, base. A node of our gossip also runs base, so a gossip.Node is accepted by base.Join:

Step 3

# package main
use example.com/tutorial/gossip
use std/sim

define main
  sim.Seed 15
  set nodes to (sim.Cluster of 5 running gossip)
  gossip.Meet nodes
  sim.StopAt 12s

# package gossip
use github.com/synodlang/examples/gossip as base

# Join every node to the first one
define Meet [ns: list of Node]
  each i, n in ns
    if (i more than 0)
      base.Join n to (item 0 of ns)

module example.com/tutorial
synod 0.2
require github.com/synodlang/examples v0.2.1

Step 4 reads what the base package knows, through our own node: n.base.Known and n.base.Down are exported fields of the package we imported. The phrase Alive at gives the number of nodes a node believes are alive, and the demo crashes one node and asks for the count at the end:

Step 4

# package main
use example.com/tutorial/gossip
use std/lesson
use std/sim
use std/text

define main
  sim.Seed 15
  set nodes to (sim.Cluster of 5 running gossip)
  gossip.Meet nodes
  after 6s
    sim.Crash (item 2 of nodes)
  after 14s
    lesson.Note (text.Join {"n1 believes ", (text.From (gossip.Alive at (item 0 of nodes))), " of 5 nodes are alive"})
  sim.StopAt 16s

# package gossip
use github.com/synodlang/examples/gossip as base
use std/lists
use std/maps

# Join every node to the first one
define Meet [ns: list of Node]
  each i, n in ns
    if (i more than 0)
      base.Join n to (item 0 of ns)

# How many nodes n believes are alive
define Alive at [n: Node]
  result ((maps.Size n.base.Known) minus (lists.Size n.base.Down))

Notice what we did not write: a single line of the gossip algorithm. The package is somebody else's, versioned and pinned, and it still animates like your own.

What to watch

Step 1: the answer travels from the server to the client, and the client's card gets the text. Step 2: the same picture, with the rules active. Steps 3 and 4: the nodes exchange heartbeats; after the crash, the others learn that n3 is down, one after another at their own times, and the caption at the end says four of five are alive.

Why it looks like this

use … as is for the collision of two last segments, and for nothing else. Renaming an import without a collision is refused, because it would give one package two names:

gossip/gossip.synod:4:17: error[unneeded-alias]: "as table" is only for imports that share a last segment
  |
4 | use std/maps as table
  |                 ^^^^^
  |
help: write "use std/maps"
  |
4 - use std/maps as table
4 + use std/maps
  |

Error unneeded-alias

synod check says error[unneeded-alias] and hints at the plain use.

Exercises

In step 1 deliver the Ask to item 0 of nodes with Of (item 0 of nodes), so that n1 asks itself. What does Heard hold?

Expected

The same text: a node can ask itself, because sending to yourself goes through the network like any other message.

In step 4 crash two nodes. What does the caption say?

Expected

The count falls to 3 of 5 once both are noticed as down.

Remove as base and rename the uses of base to gossip. What does synod check say?

Expected

error[duplicate-name]: the qualifier gossip is already the name of this package, and the hint suggests as <name>.