Stream multipart uploads from a pipe with python requests

With requests-toolbelt you can stream files to a multipart POST, but you can not stream from a pipe, as there is no way to compute the Content-Length HTTP header.

However, if you are in the situation, that you do know the length and data is coming from a pipe, you can wrap it with something like this:

class NotAPipe():
	""" Implement enough of a file like object, that MultipartEncoder can
	use a pipe to stream data """

	def __init__ ( self, pipe, length ):
		self._pipe = pipe
		self._length = length

		self._left = self._length
		self._hash = hashlib.md5()

	def read( self, *args, **kwargs ):
		data = self._pipe.read( *args, **kwargs )
		self._hash.update( data )
		self._left -= len( data )
		return data

	def md5( self ):
		return self._hash.hexdigest()

	def __len__( self ):
		return self._left

The md5 part, is optional, but plays well together with this test server, a Mojolicious::Lite application heaily based on their Streaming Cookbook example. Run it with morbo server.pl. Use it with this complete test client.