ThinkingElixir 031: Crawling the Web using Elixir with Oleg Tarasenko and Tze Yiing
In episode 31 of Thinking Elixir, we talk with Oleg Tarasenko and Tze Yiing about crawling the web using Elixir. Oleg created the crawly project to help solve this problem and Tze Yiing joined him as a contributor and maintainer. We cover how Elixir is well suited to orchestrate crawling projects, how to deal with login pages, understanding the legal concerns, building a codeless scraper and much more!
Read next How Absinthe Uses Compilation Callbacks for Schema Validation in Elixir