Stream multipart uploads from a pipe with python requests
With requests-toolbelt you can
stream files to a multipart POST, but you can not stream from a pipe, as there
is no way to compute the Content-Length
HTTP header.
However, if you are in the situation, that you do know the length and data is coming from a pipe, you can wrap it with something like this:
class NotAPipe():
""" Implement enough of a file like object, that MultipartEncoder can
use a pipe to stream data """
def __init__ ( self, pipe, length ):
self._pipe = pipe
self._length = length
self._left = self._length
self._hash = hashlib.md5()
def read( self, *args, **kwargs ):
data = self._pipe.read( *args, **kwargs )
self._hash.update( data )
self._left -= len( data )
return data
def md5( self ):
return self._hash.hexdigest()
def __len__( self ):
return self._left
The md5 part, is optional, but plays well together with this
test server, a Mojolicious::Lite
application heaily based on their
Streaming Cookbook example. Run it with morbo server.pl
. Use it with this complete test client.