Elixir Streams to process large HTTP responses on the fly
Elixir Streams to process large HTTP responses on the fly
Wouldn’t be great to process a large HTTP response, in chunks, with Elixir streams? Something like:
HTTPStream.get("https://.../large_image.tiff")
|> StreamGzip.gzip
|> Stream.into(File.stream!("image.tiff.gz"))
|> Stream.run
In this first of a two-parts article, we see how to transform an HTTPoison async response into an Elixir Stream, to easily process large remote files on the fly, gaining code clarity and reusability.
Read next TIL Ecto Constraints vs. Validations