Module implementing DownloadRequest class

class*, url=None, headers=None, request_type=<RequestType.GET: 'GET'>, post_values=None, use_session=False, data_type=<MimeType.RAW: 'raw'>, save_response=False, data_folder=None, filename=None, return_data=True, **properties)[source]

Bases: object

A class defining a single download request.

The class is a container with all parameters needed to execute a single download request and save or return downloaded data.

  • url (str or None) – An URL from where to download
  • headers (dict or None) – Headers of a HTTP request
  • request_type (str or RequestType) – Type of request, either GET or POST. Default is RequestType.GET
  • post_values (dict or None) – A dictionary of values that will be sent with a POST request. Default is None
  • use_session (bool) – A flag that specifies if the download request will require a session header
  • data_type (MimeType or str) – An expected file format of downloaded data. Default is MimeType.RAW
  • save_response (bool) – A flag defining if the downloaded data will be saved to disk. Default is False.
  • data_folder (str or None) – A folder path where the fetched data will be (or already is) saved. Default is None
  • filename (str or None) – A custom filename where the data will be saved. By default data will be saved in a folder which name are hashed request parameters.
  • return_data (bool) – A flag defining if the downloaded data will be returned as an output of download procedure. Default is True.
  • properties – Any additional parameters.

Method that raises an error if something is wrong with request parameters


Provides parameters that define the request in form of a dictionary

Parameters:include_metadata (bool) – A flag defining if also metadata parameters should be included, such as headers and current time
Returns:A dictionary of parameters
Return type:dict

It takes request url and payload and calculates a unique hashed string from them.

Returns:A hashed string
Return type:str

A method that calculates file paths relative to data_folder

Returns:Returns a pair of file paths, a request payload path and a response path. If request path is not defined it returns None.
Return type:(str or None, str)

A method that calculates file paths where request payload and response will be saved.

Returns:Returns a pair of file paths, a request payload path and a response path. Each of them can also be None if it is not defined.
Return type:(str or None, str or None)