pypath.share.curl.Curl

class pypath.share.curl.Curl(url, silent=True, get=None, post=None, req_headers=None, cache=True, debug=False, outf=None, compr=None, encoding=None, files_needed=None, connect_timeout=None, timeout=None, ignore_content_length=False, init_url=None, init_fun='get_jsessionid', init_use_cache=False, follow=True, large=False, default_mode='r', override_post=False, init_headers=False, return_headers=False, compressed=False, binary_data=None, write_cache=True, force_quote=False, sftp_user=None, sftp_passwd=None, sftp_passwd_file='.secrets', sftp_port=22, sftp_host=None, sftp_ask=None, setup=True, call=True, process=True, retries=None, cache_dir=None, bypass_url_encoding=False, empty_attempt_again=True, keep_failed=False, alpn=True, slow=False, http2=True)

Bases: FileOpener

This class is a wrapper around pycurl. You can set a vast amount of parameters. In addition it has a caching functionality: using this downloads of databases/resources is performed only once. It handles HTTP, FTP, cookies, headers, GET and POST params, multipart/form data, URL quoting, redirects, timeouts, retries, encodings, debugging. It returns either downloaded data, file pointer, files extracted from archives (gzip, tar.gz, zip). It is able to show a progress and status indicator on the console.

__init__(url, silent=True, get=None, post=None, req_headers=None, cache=True, debug=False, outf=None, compr=None, encoding=None, files_needed=None, connect_timeout=None, timeout=None, ignore_content_length=False, init_url=None, init_fun='get_jsessionid', init_use_cache=False, follow=True, large=False, default_mode='r', override_post=False, init_headers=False, return_headers=False, compressed=False, binary_data=None, write_cache=True, force_quote=False, sftp_user=None, sftp_passwd=None, sftp_passwd_file='.secrets', sftp_port=22, sftp_host=None, sftp_ask=None, setup=True, call=True, process=True, retries=None, cache_dir=None, bypass_url_encoding=False, empty_attempt_again=True, keep_failed=False, alpn=True, slow=False, http2=True)

Make this instance a logger.

Parameters:

• name – The label of this instance that will be prepended to all messages it sends to the logger.

• module – Send the messages by the logger of this module.

Methods

__init__(url[, silent, get, post, ...])	Make this instance a logger.
ask_passwd([use_passwd_file])
bytes_prefix(b)
cache_dir_exists()
cache_path(**kwargs)	Returns the cache path without performing download or creating file.
close()	Closes all file objects.
construct_binary_data()	The binary data content of a form/multipart type request can be constructed from a list of tuples (<field name>, <field value>), where field name and value are both type of bytes.
copy_file()
curl_call()
curl_init([url])
curl_progress_setup()
curl_setup([url])
decode_result()
delete_cache_file()
extract()	Calls the extracting method for compressed files.
extract_file()
get_cache_file_name()
get_hash()
get_headers()
get_jsessionid()
get_result_type()
get_type()
guess_encoding()
init_cache()
init_request()
is_quoted(string)	From http://stackoverflow.com/questions/ 1637762/test-if-string-is-url-encoded-in-php
is_quoted_plus(string)
iterfile(fileobj)
open()	Opens the file if exists.
open_file()
open_gz()
open_plain()
open_tgz()	Extracts files from tar gz.
open_zip()
print_debug_info(typ, msg)
print_status(status)
process_file()
process_url()
progress_setup()
reload()
remove_target()
replace_forbidden(name[, repl])	Replaces the characters that are forbidden in certain file systems.
report_ready()
select_cache_file()
set_binary_data()	Set binary data to be transmitted attached to POST request.
set_compressed()
set_debug()
set_get()
set_post()
set_req_headers()
set_resp_headers()
set_target()
set_title()
set_url([url])
sftp_call()
sftp_download()
sftp_url()
show_cache()
terminate_progress()
transcode()
unicode2bytes(string[, encoding])
update_progress(download_total, downloaded, ...)
url_fix([charset])	From http://stackoverflow.com/a/121017/854988

Attributes

FORBIDDEN_CHARS

classmethod cache_path(**kwargs) → str

Returns the cache path without performing download or creating file.

Parameters:

kwargs – Arguments to Curl.

close()

Closes all file objects.

construct_binary_data()

The binary data content of a form/multipart type request can be constructed from a list of tuples (<field name>, <field value>), where field name and value are both type of bytes.

extract()

Calls the extracting method for compressed files.

is_quoted(string)

From http://stackoverflow.com/questions/ 1637762/test-if-string-is-url-encoded-in-php

open()

Opens the file if exists.

open_tgz()

Extracts files from tar gz.

classmethod replace_forbidden(name: str, repl: str = '_') → str

Replaces the characters that are forbidden in certain file systems.

The slash is forbidden in Unix, while many other characters in Windows environments.

set_binary_data()

Set binary data to be transmitted attached to POST request.

binary_data is either a bytes string, or a filename, or a list of key-value pairs of a multipart form.

url_fix(charset='utf-8')

From http://stackoverflow.com/a/121017/854988