scripts.maintenance utility scripts

Maintenance scripts

Submodules

scripts.maintenance.cache script

This script runs commands on each entry in the API caches

Syntax:

python pwb.py cache [-password] [-delete] [-c "..."] [-o "..."] [dir ...]

If no directory are specified, it will detect the API caches.

If no command is specified, it will print the filename of all entries. If only -delete is specified, it will delete all entries.

The following parameters are supported:

-delete           Delete each command filtered. If that option is set the
                  default output will be nothing.

-c                Filter command in python syntax. It must evaluate to True to
                  output anything.

-o                Output command which is output when the filter evaluated to
                  True. If it returns None it won't output anything.

Examples

Print the filename of any entry with ‘wikidata’ in the key:

-c "wikidata" in entry._uniquedescriptionstr()

Customised output if the site code is ‘ar’:

-c entry.site.code == "ar"
-o uniquedesc(entry)

Or the state of the login:

-c entry.site._loginstatus == LoginStatus.NOT_ATTEMPTED
-o uniquedesc(entry)

If the function only uses one parameter for the entry it can be omitted:

-c has_password
-o uniquedesc

Available filter commands:

has_password(entry)
is_logout(entry)
empty_response(entry)
not_accessed(entry)
incorrect_hash(entry)
older_than_one_day(entry)
recent(entry)

There are helper functions which can be part of a command:

older_than(entry, interval)
newer_than(entry, interval)

Available output commands:

uniquedesc(entry)
class scripts.maintenance.cache.CacheEntry(directory, filename)[source]

Bases: pywikibot.data.api.CachedRequest

A Request cache entry.

parse_key()[source]

Parse the key loaded from the cache entry.

exception scripts.maintenance.cache.ParseError[source]

Bases: Exception

Error parsing.

scripts.maintenance.cache.empty_response(entry)[source]

Entry has no data.

scripts.maintenance.cache.has_password(entry)[source]

Entry has a password in the entry.

scripts.maintenance.cache.incorrect_hash(entry)[source]

Incorrect hash.

scripts.maintenance.cache.is_logout(entry)[source]

Entry is a logout entry.

scripts.maintenance.cache.main()[source]

Process command line arguments and invoke bot.

scripts.maintenance.cache.newer_than(entry, interval)[source]

Find newer entries.

scripts.maintenance.cache.not_accessed(entry)[source]

Entry has never been accessed.

scripts.maintenance.cache.older_than(entry, interval)[source]

Find older entries.

scripts.maintenance.cache.older_than_one_day(entry)[source]

Find more than one day old entries.

scripts.maintenance.cache.parameters(entry)[source]

Return a pretty formatted parameters list.

scripts.maintenance.cache.process_entries(cache_path, func, use_accesstime=None, output_func=None, action_func=None)[source]

Check the contents of the cache.

This program tries to use file access times to determine whether cache files are being used. However file access times are not always usable. On many modern filesystems, they have been disabled. On Unix, check the filesystem mount options. You may need to remount with ‘strictatime’.

Parameters

use_accesstime (bool tristate: - None = detect - False = don't use - True = always use) – Whether access times should be used.

scripts.maintenance.cache.recent(entry)[source]

Find entries newer than on hour.

scripts.maintenance.cache.uniquedesc(entry)[source]

Return the unique description string.

scripts.maintenance.colors script

Utility to show pywikibot colors

scripts.maintenance.colors.main()[source]

Main function.

scripts.maintenance.make_i18n_dict script

Generate an i18n file from a given script

run IDLE at topmost level:: >>> import pwb >>> from scripts.maintenance.make_i18n_dict import i18nBot >>> bot = i18nBot(‘<scriptname>’, ‘<msg dict>’) >>> bot.run()

If you have more than one message dictionary, give all these names to the bot:: >>> bot = i18nBot(‘<scriptname>’, ‘<msg dict1>’, ‘<msg dict2>’, ‘<msg dict3>’)

If you want to rename the message index use keyword arguments. This may be mixed with preleading positonal arguments:: >>> bot = i18nBot(‘<scriptname>’, ‘<msg dict1>’, the_other_msg=’<msg dict2>’)

If you have the messages as instance constants you may call the bot as follows:: >>> bot = i18nBot( … ‘<scriptname>.<class instance>’, ‘<msg dict1>’, ‘<msg dict2>’)

It’s also possible to make json files too by using to_json method after instantiating the bot. It also calls bot.run() to create the dictionaries:: >>> bot.to_json()

class scripts.maintenance.make_i18n_dict.i18nBot(script, *args, **kwargs)[source]

Bases: object

I18n bot.

print_all()[source]

Pretty print the dict as a file content to screen.

read(oldmsg, newmsg=None)[source]

Read a single message from source script.

run(quiet=False)[source]

Run the bot, read the messages from source and print the dict.

Parameters

quiet (bool) – print the result if False

to_json(quiet=True)[source]

Run the bot and create json files.

Parameters

quiet (bool) – Print the result if False

scripts.maintenance.preload_sites script

Script that preloads site and user info for all sites of given family

The following parameters are supported:

-worker:<num>     The number of parallel tasks to be run. Default is the
                  number of precessors on the machine

Usage:

python pwb.py preload_sites [{<family>}] [-worker{<num>}]

To force preloading, change the global expiry value to 0:

python pwb.py -API_config_expiry:0 preload_sites [{<family>}]
scripts.maintenance.preload_sites.preload_families(families, worker)[source]

Preload all sites of all given family files.

scripts.maintenance.preload_sites.preload_family(family, executor)[source]

Preload all sites of a single family file.

scripts.maintenance.sorting_order script

Script that updates interwiki sorting order in family.py file

scripts.maintenance.sorting_order.main()[source]

Main entry function.

scripts.maintenance.sorting_order.update_family(list_name, page)[source]

Update family.py file.

scripts.maintenance.update_linktrails script

Script that updates the linktrails in family.py file

linktrails contains a regex for each site code which holds letters that can follow a wikilink and are regarded as part of this link. This depends on the linktrail setting in LanguageXx.php. This maintenance script retrieves the site settings from wikipedia family and updates the Family linktrails dict.

scripts.maintenance.update_linktrails.coroutine(func)[source]

Decorator which starts coroutine.

scripts.maintenance.update_linktrails.format_string(code: str, pattern: str)str[source]

Format a single pattern line.

scripts.maintenance.update_linktrails.update_family_file(*args, **kwargs)[source]
scripts.maintenance.update_linktrails.update_line(*args, **kwargs)[source]
scripts.maintenance.update_linktrails.update_linktrails(family)[source]

Update linktrails for given family.

scripts.maintenance.update_linktrails.update_matched_line(*args, **kwargs)[source]
scripts.maintenance.update_linktrails.update_sites(*args, **kwargs)[source]

scripts.maintenance.wikimedia_sites script

Script that updates the language lists in Wikimedia family files

Usage:

python pwb.py wikimedia_sites [ {<family>} ]
scripts.maintenance.wikimedia_sites.update_family(families)[source]

Update family files.