pywikibot.comms package¶
Communication layer.
Submodules¶
pywikibot.comms.eventstreams module¶
Server-Sent Events client.
This file is part of the Pywikibot framework.
This module requires sseclient to be installed:
pip install sseclient
New in version 3.0.
-
class
pywikibot.comms.eventstreams.
EventStreams
(stream='[deprecated name of streams]', **kwargs)[source]¶ Bases:
object
Basic EventStreams iterator class for Server-Sent Events (SSE) protocol.
It provides access to arbitrary streams of data including recent changes. It replaces rcstream.py implementation.
Usage:
>>> stream = EventStreams(streams='recentchange') >>> stream.register_filter(type='edit', wiki='wikidatawiki') >>> change = next(iter(stream)) >>> print('{type} on page {title} by {user}.'.format_map(change)) edit on page Q32857263 by XXN-bot. >>> change {'comment': '/* wbcreateclaim-create:1| */ [[Property:P31]]: [[Q4167836]]', 'wiki': 'wikidatawiki', 'type': 'edit', 'server_name': 'www.wikidata.org', 'server_script_path': '/w', 'namespace': 0, 'title': 'Q32857263', 'bot': True, 'server_url': 'https://www.wikidata.org', 'length': {'new': 1223, 'old': 793}, 'meta': {'domain': 'www.wikidata.org', 'partition': 0, 'uri': 'https://www.wikidata.org/wiki/Q32857263', 'offset': 288986585, 'topic': 'eqiad.mediawiki.recentchange', 'request_id': '1305a006-8204-4f51-a27b-0f2df58289f4', 'schema_uri': 'mediawiki/recentchange/1', 'dt': '2017-07-13T10:55:31+00:00', 'id': 'ca13742b-67b9-11e7-935d-141877614a33'}, 'user': 'XXN-bot', 'timestamp': 1499943331, 'patrolled': True, 'id': 551158959, 'minor': False, 'revision': {'new': 518751558, 'old': 517180066}} >>> del stream
- Keyword Arguments
site – a project site object. Used when no url is given
since – a timestamp for older events; there will likely be between 7 and 31 days of history available but is not guaranteed. It may be given as a pywikibot.Timestamp, an ISO 8601 string or a mediawiki timestamp string.
streams – event stream types. Mandatory when no url is given. Multiple streams may be given as a string with comma separated stream types or an iterable of strings Refer https://stream.wikimedia.org/?doc for available Wikimedia stream types.
timeout – a timeout value indication how long to wait to send data before giving up
url – an url retrieving events from. Will be set up to a default url using _site.family settings, stream types and timestamp
- Parameters
kwargs – keyword arguments passed to SSEClient and requests lib
- Raises
ImportError – sseclient is not installed
NotImplementedError – no stream types specified
-
register_filter
(*args, **kwargs)[source]¶ Register a filter.
Filter types:
There are 3 types of filter: ‘all’, ‘any’ and ‘none’. The filter type must be given with the keyword argument ‘ftype’ (see below). If no ‘ftype’ keyword argument is given, ‘all’ is assumed as default.
You may register multiple filters for each type of filter. The behaviour of filter type is as follows:
‘none’: Skip if the any filter matches. Otherwise check ‘all’.
‘all’: Skip if not all filter matches. Otherwise check ‘any’:
‘any’: Skip if no given filter matches. Otherwise pass.
Filter functions:
Filter may be specified as external function methods given as positional argument like:
def foo(data): return True register_filter(foo, ftype='any')
The data dict from event is passed to the external filter function as a parameter and that method must handle it in a proper way and return
True
if the filter matches andFalse
otherwise.Filter keys and values:
Another method to register a filter is to pass pairs of keys and values as keyword arguments to this method. The key must be a key of the event data dict and the value must be any value or an iterable of values the
data['key']
may match or be part of it. Samples:register_filter(server_name='de.wikipedia.org') # 1 register_filter(type=('edit', 'log')) # 2 register_filter(ftype='none', bot=True) # 3
Explanation for the result of the filter function: 1.
return data['sever_name'] == 'de.wikipedia.org'
2.return data['type'] in ('edit', 'log')
3.return data['bot'] is True
- Keyword Arguments
ftype – The filter type, one of ‘all’, ‘any’, ‘none’. Default value is ‘all’
- Parameters
args (callable) – You may pass your own filter functions here. Every function should be able to handle the data dict from events.
kwargs (str, list, tuple or other sequence) – Any key returned by event data with an event data value for this given key.
- Raises
TypeError – A given args parameter is not a callable.
-
set_maximum_items
(value: int)[source]¶ Set the maximum number of items to be retrieved from the stream.
If not called, most queries will continue as long as there is more data to be retrieved from the stream.
- Parameters
value – The value of maximum number of items to be retrieved in total to set.
-
streamfilter
(data: dict)[source]¶ Filter function for eventstreams.
See the description of register_filter() how it works.
- Parameters
data – event data dict used by filter functions
-
property
url
¶ Get the EventStream’s url.
- Raises
NotImplementedError – no stream types specified
-
pywikibot.comms.eventstreams.
site_rc_listener
(site, total: Optional[int] = None)[source]¶ Yield changes received from EventStream.
- Parameters
site (Pywikibot.BaseSite) – the Pywikibot.Site object to yield live recent changes for
total – the maximum number of changes to return
- Returns
pywikibot.comms.eventstream.rc_listener configured for given site
- Raises
ImportError – sseclient installation is required
pywikibot.comms.http module¶
Basic HTTP access interface.
This module handles communication between the bot and the HTTP threads.
- This module is responsible for
Setting up a connection pool
Providing a (blocking) interface for HTTP requests
Translate site objects with query strings into URLs
URL-encoding all data
Basic HTTP error handling
This module creates and uses its own requests.Session
object.
The session is closed if the module terminates.
If required you can use your own Session object passing it to the
http.session
variable:
from pywikibot.comms import http
session = requests.Session()
http.session = session
flush()
can be called to close the session object.
-
pywikibot.comms.http.
error_handling_callback
(response)[source]¶ Raise exceptions and log alerts.
- Parameters
response (
requests.Response
) – Response returned by Session.request().
-
pywikibot.comms.http.
fetch
(uri: str, method: str = 'GET', headers: Optional[dict] = None, default_error_handling: bool = True, use_fake_user_agent: Union[bool, str] = False, body='[deprecated name of data]', **kwargs)[source]¶ HTTP request.
See
requests.Session.request
for parameters.- Parameters
uri – URL to send
method – HTTP method of the request (default: GET)
headers – dictionary of headers of the request
default_error_handling – Use default error handling
use_fake_user_agent – Set to True to use fake UA, False to use pywikibot’s UA, str to specify own UA. This behaviour might be overridden by domain in config.
- Keyword Arguments
charset – Either a valid charset (usable for str.decode()) or None to automatically chose the charset from the returned header (defaults to latin-1)
verify – verify the SSL certificate (default is True)
callbacks – Methods to call once data is fetched
- Return type
requests.Response
-
pywikibot.comms.http.
flush
()[source]¶ Close the session object. This is called when the module terminates.
-
pywikibot.comms.http.
get_authentication
(uri: str) → Optional[tuple][source]¶ Retrieve authentication token.
- Parameters
uri – the URI to access
- Returns
authentication token
-
pywikibot.comms.http.
request
(site, uri: Optional[str] = None, headers: Optional[dict] = None, body='[deprecated name of data]', **kwargs)[source]¶ Request to Site with default error handling and response decoding.
See
requests.Session.request
for additional parameters.The optional uri is a relative uri from site base uri including the document root ‘/’.
- Parameters
site (pywikibot.site.BaseSite) – The Site to connect to
uri – the URI to retrieve
- Keyword Arguments
charset – Either a valid charset (usable for str.decode()) or None to automatically chose the charset from the returned header (defaults to latin-1)
- Returns
The received data Response
-
pywikibot.comms.http.
user_agent
(site=None, format_string: Optional[str] = None) → str[source]¶ Generate the user agent string for a given site and format.
- Parameters
site (BaseSite) – The site for which this user agent is intended. May be None.
format_string – The string to which the values will be added using str.format. Is using config.user_agent_format when it is None.
- Returns
The formatted user agent
-
pywikibot.comms.http.
user_agent_username
(username=None)[source]¶ Reduce username to a representation permitted in HTTP headers.
To achieve that, this function: 1) replaces spaces (‘ ‘) with ‘_’ 2) encodes the username as ‘utf-8’ and if the username is not ASCII 3) URL encodes the username if it is not ASCII, or contains ‘%’