requests-toolbelt
A utility belt for advanced users of python-requests
Downloads: 0 (30 days)
Description
The Requests Toolbelt
=====================
This is just a collection of utilities for `python-requests`_, but don't
really belong in ``requests`` proper. The minimum tested requests version is
``2.1.0``. In reality, the toolbelt should work with ``2.0.1`` as well, but
some idiosyncracies prevent effective or sane testing on that version.
``pip install requests-toolbelt`` to get started!
multipart/form-data Encoder
---------------------------
The main attraction is a streaming multipart form-data object, ``MultipartEncoder``.
Its API looks like this:
.. code-block:: python
from requests_toolbelt import MultipartEncoder
import requests
m = MultipartEncoder(
fields={'field0': 'value', 'field1': 'value',
'field2': ('filename', open('file.py', 'rb'), 'text/plain')}
)
r = requests.post('http://httpbin.org/post', data=m,
headers={'Content-Type': m.content_type})
You can also use ``multipart/form-data`` encoding for requests that don't
require files:
.. code-block:: python
from requests_toolbelt import MultipartEncoder
import requests
m = MultipartEncoder(fields={'field0': 'value', 'field1': 'value'})
r = requests.post('http://httpbin.org/post', data=m,
headers={'Content-Type': m.content_type})
Or, you can just create the string and examine the data:
.. code-block:: python
# Assuming `m` is one of the above
m.to_string() # Always returns unicode
User-Agent constructor
----------------------
You can easily construct a requests-style ``User-Agent`` string::
from requests_toolbelt import user_agent
headers = {
'User-Agent': user_agent('my_package', '0.0.1')
}
r = requests.get('https://api.github.com/users', headers=headers)
SSLAdapter
----------
The ``SSLAdapter`` was originally published on `Cory Benfield's blog`_.
This adapter allows the user to choose one of the SSL protocols made available
in Python's ``ssl`` module for outgoing HTTPS connections:
.. code-block:: python
from requests_toolbelt import SSLAdapter
import requests
import ssl
s = requests.Session()
s.mount('https://', SSLAdapter(ssl.PROTOCOL_TLSv1))
cookies/ForgetfulCookieJar
--------------------------
The ``ForgetfulCookieJar`` prevents a particular requests session from storing
cookies:
.. code-block:: python
from requests_toolbelt.cookies.forgetful import ForgetfulCookieJar
session = requests.Session()
session.cookies = ForgetfulCookieJar()
Contributing
------------
Please read the `suggested workflow
<https://toolbelt.readthedocs.io/en/latest/contributing.html>`_ for
contributing to this project.
Please report any bugs on the `issue tracker`_
.. _Cory Benfield's blog: https://lukasa.co.uk/2013/01/Choosing_SSL_Version_In_Requests/
.. _python-requests: https://github.com/kennethreitz/requests
.. _issue tracker: https://github.com/requests/toolbelt/issues
History
=======
1.0.0 -- 2023-05-01
-------------------
Breaking Changes
~~~~~~~~~~~~~~~~
- Removed Google App Engine support to allow using urllib3 2.0
Fixed Bugs
~~~~~~~~~~
- Ensured the test suite no longer reaches the Internet
Miscellaneous
~~~~~~~~~~~~~
- Added explicit support for Python 3.11
0.10.1 -- 2022-10-25
--------------------
Fixed Bugs
~~~~~~~~~~
- Fix urllib3 warning to only emit on X509Adapter usage
0.10.0 -- 2022-10-06
--------------------
New Features
~~~~~~~~~~~~
- Add support for preparing requests in BaseUrlSession
Fixed Bugs
~~~~~~~~~~
- Fixing missing newline in dump utility
0.9.1 -- 2019-01-29
-------------------
Fixed Bugs
~~~~~~~~~~
- Fix import of pyOpenSSL shim from urllib3 for PKCS12 adapter
0.9.0 -- 2019-01-29
-------------------
New Features
~~~~~~~~~~~~
- Add X509 Adapter that can handle PKCS12
- Add stateless solution for streaming files by MultipartEncoder from one host to another (in chunks)
Fixed Bugs
~~~~~~~~~~
- Update link to example
- Move import of ``ABCs`` from collections into version-specific part of
_compat module
- Fix backwards incompatibility in ``get_encodings_from_content``
- Correct callback documentation for ``MultipartEncoderMonitor``
- Fix bug when ``MultipartEncoder`` is asked to encode zero parts
- Correct the type of non string request body dumps
- Removed content from being stored in MultipartDecoder
- Fix bug by enabling support for contenttype with capital letters.
- Coerce proxy URL to bytes before dumping request
- Avoid bailing out with exception upon empty response reason
- Corrected Pool documentation
- Corrected parentheses match in example usage
- Fix "oject" to "object" in ``MultipartEncoder``
- Fix URL for the project after the move
- Add fix for OSX TCPKeepAliveAdapter
Miscellaneous
~~~~~~~~~~~~~
- Remove py33 from testing and add Python 3.6 and nightly testing to the travis matrix.
0.8.0 -- 2017-05-20
-------------------
More information about this release can be found on the `0.8.0 milestone`_.
New Features
~~~~~~~~~~~~
- Add ``UserAgentBuilder`` to provide more control over generated User-Agent
strings.
Fixed Bugs
~~~~~~~~~~
- Include ``_validate_certificate`` in the lits of picked attributes on the
``AppEngineAdapter``.
- Fix backwards incompatibility in ``get_encodings_from_content``
.. _0.8.0 milestone:
https://github.com/requests/toolbelt/milestones/0.8.0
0.7.1 -- 2017-02-13
-------------------
More information about this release can be found on the `0.7.1 milestone`_.
Fixed Bugs
~~~~~~~~~~
- Fixed monkey-patching for the AppEngineAdapter.
- Make it easier to disable certificate verification when monkey-patching
AppEngine.
- Handle ``multipart/form-data`` bodies without a trailing ``CRLF``.
.. links
.. _0.7.1 milestone:
https://github.com/requests/toolbelt/milestone/9
0.7.0 -- 2016-07-21
-------------------
More information about this release can be found on the `0.7.0 milestone`_.
New Features
~~~~~~~~~~~~
- Add ``BaseUrlSession`` to allow developers to have a session that has a
"Base" URL. See the documentation for more details and examples.
- Split the logic of ``stream_response_to_file`` into two separate functions:
* ``get_download_file_path`` to generate the file name from the Response.
* ``stream_response_to_file`` which will use ``get_download_file_path`` if
necessary
Fixed Bugs
~~~~~~~~~~
- Fixed the issue for people using *very* old versions of Requests where they
would see an ImportError from ``requests_toolbelt._compat`` when trying to
import ``connection``.
.. _0.7.0 milestone:
https://github.com/requests/toolbelt/milestones/0.7.0
0.6.2 -- 2016-05-10
-------------------
Fixed Bugs
~~~~~~~~~~
- When passing a timeout via Requests, it was not appropriately translated to
the timeout that the urllib3 code was expecting.
0.6.1 -- 2016-05-05
-------------------
Fixed Bugs
~~~~~~~~~~
- Remove assertion about request URLs in the AppEngineAdapter.
- Prevent pip from installing requests 3.0.0 when that is released until we
are ready to handle it.
0.6.0 -- 2016-01-27
-------------------
More information about this release can be found on the `0.6.0 milestone`_.
New Features
~~~~~~~~~~~~
- Add ``AppEngineAdapter`` to support developers using Google's AppEngine
platform with Requests.
- Add ``GuessProxyAuth`` class to support guessing between Basic and Digest
Authentication for proxies.
Fixed Bugs
~~~~~~~~~~
- Ensure that proxies use the correct TLS version when using the
``SSLAdapter``.
- Fix an ``AttributeError`` when using the ``HTTPProxyDigestAuth`` class.
Miscellaneous
~~~~~~~~~~~~~
- Drop testing support for Python 3.2. virtualenv and pip have stopped
supporting it meaning that it is harder to test for this with our CI
infrastructure. Moving forward we will make a best-effort attempt to
support 3.2 but will not test for it.
.. _0.6.0 milestone:
https://github.com/requests/toolbelt/milestones/0.6.0
0.5.1 -- 2015-12-16
-------------------
More information about this release can be found on the `0.5.1 milestone`_.
Fixed Bugs
~~~~~~~~~~
- Now papers over the differences in requests' ``super_len`` function from
versions prior to 2.9.0 and versions 2.9.0 and later.
.. _0.5.1 milestone:
https://github.com/requests/toolbelt/milestones/0.5.1
0.5.0 -- 2015-11-24
-------------------
More information about this release can be found on the `milestone
<https://github.com/requests/toolbelt/issues?utf8=%E2%9C%93&q=is%3Aall+milestone%3A0.5+>`_
for 0.5.0.
New Features
~~~~~~~~~~~~
- The ``tee`` submodule was added to ``requests_toolbelt.downloadutils``. It
allows you to iterate over the bytes of a response while also writing them
to a file. The ``tee.tee`` function, expects you to pass an open file
object, while ``tee.tee_to_file`` will use the provided file name to open
the file for you.
- Added a new parameter to ``requests_toolbelt.utils.user_agent`` that allows
the user to specify additional items.
- Added nested form-data helper,
``requests_toolbelt.utils.formdata.urlencode``.
- Added the ``ForgetfulCookieJar`` to ``requests_toolbelt.cookies``.
- Added utilities for dumping the information about a request-response cycle
in ``requests_toolbelt.utils.dump``.
- Implemented the API described in the ``requests_toolbelt.threaded`` module
docstring, i.e., added ``requests_toolbelt.threaded.map`` as an available
function.
Fixed Bugs
~~~~~~~~~~
- Now papers over the API differences in versions of requests installed from
system packages versus versions of requests installed from PyPI.
- Allow string types for ``SourceAddressAdapter``.
0.4.0 -- 2015-04-03
-------------------
For more information about this release, please see `milestone 0.4.0
<https://github.com/requests/toolbelt/issues?q=milestone%3A0.4>`_
on the project's page.
New Features
~~~~~~~~~~~~
- A naive implemenation of a thread pool is now included in the toolbelt. See
the docs in ``docs/threadi