# `Slap.Cluster.Strategy.LeaseLoop`
[🔗](https://github.com/mjrusso/slap/blob/slap_cluster-v0.6.1/slap_cluster/lib/slap/cluster/strategy/lease_loop.ex#L1)

The placement loop of `Slap.Cluster.Strategy.ObjectLease` (leases in
the object store), behind a storage behaviour for other lease stores:
one lease per shard, a heartbeat per node, fair shares, and self-fencing.
Where the leases live is a `Slap.Cluster.Strategy.LeaseLoop.Store`.

Every `:interval` (a third of `:lease_ttl` by default), each node:

  1. heartbeats;
  2. renews its leases, and stops any shard whose lease it no longer holds;
  3. computes its fair share, `ceil(shards / live nodes)`, where a node is
     live if it heartbeated within the TTL (three heartbeats), so a dead
     node stops counting when its leases expire;
  4. releases shards above its share: it stops the shard (which closes and
     flushes it) and only then frees the lease, so the next owner opens a
     closed database;
  5. reads every lease (for `lookup/2`), and claims free or expired ones up
     to its share, opening those shards with the lease's generation.

Shards open and close in a process of their own, the runner, so
renewals keep their pace however long a rebalance takes. A shard keeps
its lease (and it is renewed) while it opens or closes. After a failed
renewal the loop tries again after half the interval.

**Self-fencing.** The runner (not the loop, which may be stuck in a slow
call to the lease store) keeps a deadline per shard: its last successful
renewal or claim, plus the TTL, minus `:margin`. A shard whose deadline
passes is stopped before its lease can expire, and one still waiting to
open (`:max_concurrency` are opened at once) is not opened. Every open
and close goes to the host from the runner, so the host gets each
shard's in order: a stop always comes after the open it follows. If the
runner exits, the loop stops too, and the cluster's supervisor stops
every shard. If a node is too slow even for that (a long pause), the new
owner's open fences it in SlateDB, so there is still at most one
effective writer.

A failed read of the owners skips the claims of that round, and
`lookup/2` keeps the owners of the last successful read. An owner is
routed to only if its name is already an atom on this node (it is
connected, was connected, or is named in configuration such as a peer
discovery topology): lease data never creates atoms. A node that has
never heard of an owner treats its shards as unassigned until the nodes
connect, so the nodes need peer discovery (for example libcluster).

Options common to the lease strategies: `:lease_ttl` (ms, default
15,000), `:interval` (default a third of the TTL), `:margin` (default a sixth of
the TTL), `:name` (this node's name, default `node()`; `call/4` routes to
it), `:cluster_name` (default the cluster module's name), and
`:max_concurrency` (shards opened at once, default 16).

Telemetry: `[:slap, :cluster, :lease, :claim | :release | :lost]` with
`%{cluster, shard}`, `[:lease, :self_fence]` with `%{cluster, shards}`, and
`[:lease, :renew_failed]`.

# `__using__`
*macro* 

Makes the calling module a lease strategy whose leases it keeps itself: it
implements `Slap.Cluster.Strategy` with this loop, and must implement
`Slap.Cluster.Strategy.LeaseLoop.Store`. The strategy's options are
given to the store's `init/2` as well as to the loop.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
