client
Restful json clients.
Use Resource for a connection to a single host.
Use Pool for persistent thread-safe connections to a single host.
Use Resources for multiple hosts with simple partitioning or replication.
Use Shards for horizontally partitioning hosts by different keys.
Use Replicas in coordination with automatic host synchronization.
Resources optionally reuse connections, handling request timeouts.
Broadcasting to multiple resources is parallelized with asynchronous requests and responses.
The load balancing strategy is randomized, biased by the number of cached connections available.
This inherently provides limited failover support, but applications must still handle exceptions as desired.
Replicas will automatically retry if host is unreachable.
Note
Caching more client connections than the backend server supports may cause blocking under load. CherryPy’s default thread pool is 10.
Response
-
class lupyne.client.Response(sock, debuglevel=0, strict=0, method=None, buffering=False)[source]
Bases: httplib.HTTPResponse
A completed response which handles json and caches its body.
-
status
HTTP status code
-
reason
HTTP status message
-
body
string of entire response body
-
time
server response time
-
__nonzero__()[source]
Return whether status is successful.
-
__call__()[source]
Return evaluated response body or raise exception.
Resource
-
class lupyne.client.Resource(host, port=None, strict=None, timeout=<object object at 0x10dc33510>, source_address=None)[source]
Bases: httplib.HTTPConnection
Synchronous connection which handles json responses.
-
call(method, path, body=None, params=(), redirect=False)[source]
Send request and return completed response.
-
delete(path, **params)[source]
Return response body from DELETE request.
-
download(path, filename)[source]
Download response body from GET request to a file.
-
get(path, **params)[source]
Return response body from GET request.
-
getresponse(filename='')[source]
Return completed response, optionally write response body to a file.
-
multicall(*requests)[source]
Pipeline requests (method, path[, body]) and generate completed responses.
-
post(path, body=None, **kwargs)[source]
Return response body from POST request.
-
put(path, body=None, **kwargs)[source]
Return response body from PUT request.
-
request(method, path, body=None)[source]
Send request after handling body and headers.
Pool
-
class lupyne.client.Pool(host, limit=0)[source]
Bases: collections.deque
Thread-safe resource pool for one host.
-
call(method, path, body=None)[source]
Send request and return completed response.
-
resource_class
alias of Resource
-
stream(method, path, body=None)[source]
Generate resource, initial response, and final response while handling timeouts.
Resources
-
class lupyne.client.Resources(hosts, limit=0)[source]
Bases: dict
Thread-safe mapping of hosts to resource pools.
Parameters: |
- hosts – host[:port] strings
- limit – maximum number of cached resources per host
|
-
broadcast(method, path, body=None, hosts=())[source]
Send requests and return responses from all hosts, optionally from given subset.
-
choice(hosts)[source]
Return chosen host according to priority.
-
priority(host)[source]
Return priority for host. None may be used to eliminate from consideration.
-
unicast(method, path, body=None, hosts=())[source]
Send request and return response from any host, optionally from given subset.
Shards
-
class lupyne.client.Shards(items=(), limit=0, **multimap)[source]
Bases: dict
Mapping of keys to host clusters, with associated resources.
Parameters: |
- items – host, key pairs
- limit – maximum number of cached connections per host
- multimap – mapping of hosts to multiple keys
|
-
resources
Resources mapping.
-
broadcast(key, method, path, body=None)[source]
Send requests and return responses from all hosts for corresponding key.
-
choice(hosts)
Return chosen host according to priority.
-
multicast(keys, method, path, body=None)[source]
Send requests and return responses from a minimal subset of hosts which cover all corresponding keys.
Response overlap is possible depending on partitioning.
-
priority(hosts)[source]
Return combined priority for hosts.
-
unicast(key, method, path, body=None)[source]
Send request and return response from any host for corresponding key.
Replicas
-
class lupyne.client.Replicas(hosts, limit=0)[source]
Bases: lupyne.client.Resources
Resources which failover assuming the hosts are being automatically synchronized.
Writes are dispatched to the first host and sequentially failover.
Reads are balanced among all remaining hosts.
-
call(method, path, body=None, params=(), retry=False)[source]
Send request and return completed response, even if hosts are unreachable.
Parameters: | retry – optionally retry request on http errors as well, such as waiting for indexer promotion |
-
delete(path, **params)
Return response body from DELETE request.
-
discard(host)[source]
Remove host from availablity.
-
get(path, **params)
Return response body from GET request.
-
post(path, body=None, **kwargs)
Return response body from POST request.
-
put(path, body=None, **kwargs)
Return response body from PUT request.