etcd3.stateful.lease

Sync lease util

class etcd3.stateful.lease.Lease(client, ttl, ID=0, new=True)

Bases: object

__init__(client, ttl, ID=0, new=True)

Parameters:

• client (BaseClient) – client instance of etcd3
• ID (int) – ID is the requested ID for the lease. If ID is set to 0, the lessor chooses an ID.
• new (bool) – whether grant a new lease or maintain a exist lease by its id [default: True]

ID

Property: the id of the granted lease

Returns:int

grant()

Grant the lease if new is set to False or it just inherit the lease of the specified id

When granting new lease if ID is set to 0, the lessor will chooses an ID.

time_to_live(keys=False)

Retrieves lease information.

Parameters:keys (bool) – whether return the keys that attached to the lease

ttl()

Get the ttl that lease has left

Returns:int

alive()

Tell if the lease is still alive

Returns:bool

keepalive_once()

Call keepalive for once to refresh the ttl of the lease

refresh()

Call keepalive for once to refresh the ttl of the lease

keepalive(keep_cb=None, cancel_cb=None)

Start a daemon thread to constantly keep the lease alive

Parameters:

• keep_cb (callable) – callback function that will be called after every refresh
• cancel_cb (callable) – callback function that will be called after cancel keepalive

cancel_keepalive(join=True)

stop keeping-alive

Parameters:join (bool) – whether to wait the keepalive thread to exit

jammed()

if is failed to keepalive at the last loop

revoke()

revoke the lease

__enter__()

__exit__(exc_type, exc_val, exc_tb)

etcd3.stateful.transaction

class etcd3.stateful.transaction.Txn(client, compare=None, success=None, failure=None)

Bases: object

Txn (transaction) util provides a human friendly way to build kv.txn request Usage:

>>> from etcd3 import Client, Txn
>>> txn = Txn(Client())
>>> txn.compare(txn.key('foo').value == 'bar')
>>> txn.success(txn.put('foo', 'bra'))
>>> txn.commit()
etcdserverpbTxnResponse(header=etcdserverpbResponseHeader(cluster_id=11588568905070377092, member_id=128088275939295631, revision=15656, raft_term=4), succeeded=True, responses=[etcdserverpbResponseOp(response_put=etcdserverpbPutResponse(header=etcdserverpbResponseHeader(revision=15656)))])

From google paxosdb paper:

Our implementation hinges around a powerful primitive which we call MultiOp. All other database operations except for iteration are implemented as a single call to MultiOp. A MultiOp is applied atomically and consists of three components:

1. A list of tests called guard. Each test in guard checks a single entry in the database. It may check for the absence or presence of a value, or compare with a given value. Two different tests in the guard may apply to the same or different entries in the database. All tests in the guard are applied and MultiOp returns the results. If all tests are true, MultiOp executes t op (see item 2 below), otherwise it executes f op (see item 3 below).
2. A list of database operations called t op. Each operation in the list is either an insert, delete, or lookup operation, and applies to a single database entry. Two different operations in the list may apply to the same or different entries in the database. These operations are executed if guard evaluates to true.
3. A list of database operations called f op. Like t op, but executed if guard evaluates to false.

__init__(client, compare=None, success=None, failure=None)

Parameters:

• client (BaseClient) – the instance of etcd3’s client
• compare – guard components of the transaction, default to []
• success – success components of the transaction, default to []
• failure – failure components of the transaction, default to []

clear()

clear all ops

compare(compareOp)

Add a test to the transaction guard component

Parameters:compareOp – TxnCompareOp Returns:self

If(compareOp)

Add a test to the transaction guard component

Parameters:compareOp – TxnCompareOp Returns:self

success(successOp)

Add a database operation to the transaction success component

Then(successOp)

Add a database operation to the transaction success component

failure(failureOp)

Add a database operation to the transaction failure component

Else(failureOp)

Add a database operation to the transaction failure component

commit()

static key(key=None, range_end=None, prefix=None, all=None)

Get the TxnCompareOp of a key

Parameters:

• key (str or bytes) – key is the subject key for the comparison operation.
• range_end (str or bytes) – range_end is the upper bound on the requested range [key, range_end). If range_end is ‘’, the range is all keys >= key. If range_end is key plus one (e.g., “aa”+1 == “ab”, “aÿ”+1 == “b”), then the range request gets all keys prefixed with key. If both key and range_end are ‘’, then the range request returns all keys.
• prefix (bool) – if the key is a prefix [default: False]
• all (bool) – all the keys [default: False]

Returns:

TxnCompareOp

static range(key=None, range_end=None, limit=0, revision=None, serializable=False, keys_only=False, count_only=False, min_mod_revision=None, max_mod_revision=None, min_create_revision=None, max_create_revision=None, sort_order=<RangeRequestSortOrder.NONE: 'NONE'>, sort_target=<RangeRequestSortTarget.KEY: 'KEY'>, prefix=None, all=None)

Operation of keys in the range from the key-value store.

Parameters:

• key (str or bytes) – key is the first key for the range. If range_end is not given, the request only looks up key.
• range_end (str or bytes) – range_end is the upper bound on the requested range [key, range_end). If range_end is ‘’, the range is all keys >= key. If range_end is key plus one (e.g., “aa”+1 == “ab”, “aÿ”+1 == “b”), then the range request gets all keys prefixed with key. If both key and range_end are ‘’, then the range request returns all keys.
• limit (int) – limit is a limit on the number of keys returned for the request. When limit is set to 0, it is treated as no limit.
• revision (int) – revision is the point-in-time of the key-value store to use for the range. If revision is less or equal to zero, the range is over the newest key-value store. If the revision has been compacted, ErrCompacted is returned as a response.
• sort_order (RangeRequestSortOrder) – sort_order is the order for returned sorted results.
• sort_target (RangeRequestSortTarget) – sort_target is the key-value field to use for sorting.
• serializable (bool) – serializable sets the range request to use serializable member-local reads. Range requests are linearizable by default; linearizable requests have higher latency and lower throughput than serializable requests but reflect the current consensus of the cluster. For better performance, in exchange for possible stale reads, a serializable range request is served locally without needing to reach consensus with other nodes in the cluster.
• keys_only (bool) – keys_only when set returns only the keys and not the values.
• count_only (bool) – count_only when set returns only the count of the keys in the range.
• min_mod_revision (int) – min_mod_revision is the lower bound for returned key mod revisions; all keys with lesser mod revisions will be filtered away.
• max_mod_revision (int) – max_mod_revision is the upper bound for returned key mod revisions; all keys with greater mod revisions will be filtered away.
• min_create_revision (int) – min_create_revision is the lower bound for returned key create revisions; all keys with lesser create revisions will be filtered away.
• max_create_revision (int) – max_create_revision is the upper bound for returned key create revisions; all keys with greater create revisions will be filtered away.
• prefix (bool) – if the key is a prefix [default: False]
• all (bool) – all the keys [default: False]

static put(key, value, lease=0, prev_kv=False, ignore_value=False, ignore_lease=False)

Operation of puts the given key into the key-value store. A put request increments the revision of the key-value store and generates one event in the event history.

Parameters:

• key (str or bytes) – key is the key, in bytes, to put into the key-value store.
• value (str) – value is the value, in bytes, to associate with the key in the key-value store.
• lease (int) – lease is the lease ID to associate with the key in the key-value store. A lease value of 0 indicates no lease.
• prev_kv (bool) – If prev_kv is set, etcd gets the previous key-value pair before changing it. The previous key-value pair will be returned in the put response.
• ignore_value (bool) – If ignore_value is set, etcd updates the key using its current value. Returns an error if the key does not exist.
• ignore_lease (bool) – If ignore_lease is set, etcd updates the key using its current lease. Returns an error if the key does not exist.

static delete(key=None, range_end=None, prev_kv=False, prefix=None, all=None)

Operation of deletes the given range from the key-value store. A delete request increments the revision of the key-value store and generates a delete event in the event history for every deleted key.

Parameters:

• key (str or bytes) – key is the first key to delete in the range.
• range_end (str or bytes) – range_end is the key following the last key to delete for the range [key, range_end). If range_end is not given, the range is defined to contain only the key argument. If range_end is one bit larger than the given key, then the range is all the keys with the prefix (the given key). If range_end is ‘’, the range is all keys greater than or equal to the key argument.
• prev_kv (bool) – If prev_kv is set, etcd gets the previous key-value pairs before deleting it. The previous key-value pairs will be returned in the delete response.
• prefix (bool) – if the key is a prefix [default: False]
• all (bool) – all the keys [default: False]

clone()

Returns:Txn

class etcd3.stateful.transaction.TxnCompareOp(key, range_end=None)

Bases: object

The operator of transaction’s compare part

__init__(key, range_end=None)

Parameters:key – the key to compare

value

represents the value of the key

mod

represents the mod_revision of the key

version

represents the version of the key

create

represents the create_revision of the key

lease

represents the lease_id of the key

ref: https://github.com/etcd-io/etcd/blob/v3.3.12/clientv3/compare.go#L87-L91

LeaseValue compares a key’s LeaseID to a value of your choosing. The empty LeaseID is 0, otherwise known as NoLease.

to_compare()

return the compare payload that the rpc accepts

etcd3.stateful.watch

exception etcd3.stateful.watch.OnceTimeout

Bases: OSError

Timeout caused by watch once

class etcd3.stateful.watch.KeyValue(data)

Bases: object

Model of the key-value of the event

__init__(data)

Initialize self. See help(type(self)) for accurate signature.

get(key, default=None)

class etcd3.stateful.watch.Event(data, header=None)

Bases: etcd3.stateful.watch.KeyValue

Watch event

__init__(data, header=None)

Parameters:

• data – dict data of a etcdserverpbWatchResponse.events[<mvccpbEvent>]
• header – the header of etcdserverpbWatchResponse

class etcd3.stateful.watch.Watcher(client, max_retries=-1, key=None, range_end=None, start_revision=None, progress_notify=None, prev_kv=None, prefix=None, all=None, no_put=False, no_delete=False)

Bases: object

__init__(client, max_retries=-1, key=None, range_end=None, start_revision=None, progress_notify=None, prev_kv=None, prefix=None, all=None, no_put=False, no_delete=False)

Initialize a watcher

Parameters:

• client (BaseClient) – client instance of etcd3
• max_retries (int) – max retries when watch failed due to network problem, -1 means no limit [default: -1]
• key (str or bytes) – key is the key to register for watching.
• range_end (str or bytes) – range_end is the end of the range [key, range_end) to watch. If range_end is not given, only the key argument is watched. If range_end is equal to ‘’, all keys greater than or equal to the key argument are watched. If the range_end is one bit larger than the given key, then all keys with the prefix (the given key) will be watched.
• start_revision (int) – start_revision is an optional revision to watch from (inclusive). No start_revision is “now”.
• progress_notify (bool) – progress_notify is set so that the etcd server will periodically send a WatchResponse with no events to the new watcher if there are no recent events. It is useful when clients wish to recover a disconnected watcher starting from a recent known revision. The etcd server may decide how often it will send notifications based on current load.
• prev_kv (bool) – If prev_kv is set, created watcher gets the previous KV before the event happens. If the previous KV is already compacted, nothing will be returned.
• prefix (bool) – if the key is a prefix [default: False]
• all (bool) – all the keys [default: False]
• no_put (bool) – filter out the put events at server side before it sends back to the watcher. [default: False]
• no_delete (bool) – filter out the delete events at server side before it sends back to the watcher. [default: False]

set_default_timeout(timeout)

Set the default timeout of watch request

Parameters:timeout (int) – timeout in seconds

clear_revision()

Clear the start_revision that stored in watcher

clear_callbacks()

Remove all callbacks

request_create()

Start a watch request

request_cancel()

Cancel the watcher [Not Implemented because of etcd3 returns no watch_id]

static get_filter(filter)

Get the event filter function

Parameters:filter (callable or regex string or EventType or None) – will generate a filter function from this param Returns:callable

onEvent(filter_or_cb, cb=None)

Add a callback to a event that matches the filter

If only one param is given, which is filter_or_cb, it will be treated as the callback. If any event comes, it will be called.

Parameters:

• filter_or_cb (callable or regex string or EventType) – filter or callback function
• cb – the callback function

unEvent(filter=None, cb=None)

remove a callback or filter event that’s been previously added via onEvent() If both parameters are given they are ANDd together; to OR the, make two calls.

Parameters:

• filter (callable or regex string or EventType) – the callable filter or regex string or EventType the event to be removed was registerd with
• cb – the callback funtion the event to be removed was registerd with

dispatch_event(event)

Find the callbacks, if callback’s filter fits this event, call the callback

Parameters:event – Event

run()

Run the watcher and handel events by callbacks

stop()

Stop watching, close the watch stream and exit the daemon thread

cancel()

Stop watching, close the watch stream and exit the daemon thread

runDaemon()

Run Watcher in a daemon thread

watch_once(filter=None, timeout=None)

watch the filtered event, once have event, return it if timed out, return None

__enter__()

__exit__(exc_type, exc_val, exc_tb)

etcd3.stateful.lock

exception etcd3.stateful.lock.EtcdLockError

Bases: Exception

exception etcd3.stateful.lock.EtcdLockAcquireTimeout

Bases: Exception

class etcd3.stateful.lock.Lock(client, lock_name, lock_ttl=60, reentrant=None, lock_prefix='_locks')

Bases: object

Locking recipe for etcd, inspired by the kazoo recipe for zookeeper

DEFAULT_LOCK_TTL = 60

HOST = 'host'

PROCESS = 'process'

THREAD = 'thread'

__init__(client, lock_name, lock_ttl=60, reentrant=None, lock_prefix='_locks')

Parameters:

• client (BaseClient) – instance of etcd.Client
• lock_name (str) – the name of the lock
• lock_ttl (int) – ttl of the lock, default is 60s
• reentrant (str) – the reentrant type of the lock can set to Lock.HOST, Lock.PROCESS, Lock.THREAD
• lock_prefix (str) – the prefix of the lock key

holders()

tell how many holders are holding the lock

Returns:int

incr_holder()

Atomic increase the holder count by 1

decr_holder()

Atomic decrease the holder count by 1

is_acquired

if the lock is acquired

acquired

if the lock is acquired

acquire(block=True, lock_ttl=None, timeout=None, delete_key=True)

Acquire the lock.

Parameters:

• block (bool) – Block until the lock is obtained, or timeout is reached [default: True]
• lock_ttl (int) – The duration of the lock we acquired, set to None for eternal locks
• timeout (int) – The time to wait before giving up on getting a lock
• delete_key (bool) – whether delete the key if it has not attached to any lease [default: True]

wait(locker=None, timeout=None)

Wait until the lock is lock is able to acquire

Parameters:

• locker – kv of the lock
• timeout – wait timeout

release()

Release the lock

__enter__()

You can use the lock as a contextmanager

__exit__(type, value, traceback)