GoTrained · All posts

Scrapy: Powerful Web Scraping & Crawling with Python - Course

GoTrained · 2017-07-15

If you come here searching for a discount coupon for our best-selling Scrapy course, you finally got it. Click here to get your Scrapy course coupon. Enjoy your hassle-free learning experience and earn more income through applying for well-paid Scrapy jobs.

This Scrapy online course starts by covering the fundamentals of using Scrapy, and then concentrate on Scrapy advanced features of creating and automating web crawlers. By the end of this Scrapy course, you will be able to:

• Understand what Scrapy is, the differences between Scrapy and other Python-based web scraping libraries such as BeautifulSoup, LXML, Requests, and Selenium, and when it is better to use Scrapy.


• Create a Scrapy project and and then build a basic Spider to scrape data from a website.


• Use XPath commands with Scrapy to extract data.


• Build a more advanced Scrapy spider to iterate multiple pages of a website and scrape data from each page.


• Understand the architecture of Scrapy projects: the overall layout of a Scrapy project; what each field represents and how you can use them in your spider code.


• Apply Web Scraping best practices to avoid getting banned by the websites you are scraping.


• Deploy a Scrapy web crawler to the Scrapy Cloud platform easily. Scrapy Cloud is a platform from Scrapinghub to run, automate, and manage your web crawlers in the cloud, without the need to set up your own servers.


• How to use Scrapy for web scraping authenticated (logged in) user sessions, i.e. on websites that require a username and password before displaying data.


• Use Scrapy CrawlSpider which is the most commonly used spider for crawling regular websites, as it provides a convenient mechanism for following links by defining a set of rules. We will also use Link Extractor object which defines how links will be extracted from each crawled page; it allows us to grab all the links on a page, no matter how many of them there are.


• Combine Selenium with Scrapy to create web crawlers of dynamic web pages. When you cannot fetch data directly from the source, but you need to load the page, fill in a form, click somewhere, scroll down and so on, namely if you are trying to scrape data from a website that has a lot of AJAX calls and JavaScript execution to render webpages, it is good to use Selenium along with Scrapy.


• Export the scraping output to CSV, JSON, and XML files.


• Store the data extracted by Scrapy into MySQL and MongoDB databases.

• You will be able to download all source code explained in the course video tutorials.


• Through the Q&A board of the course, you will feel free to ask any questions you have about web scraping and crawling in general and Scrapy in particular.

Scrapy: Powerful Web Scraping & Crawling with Python - Course Coupon

https://www.udemy.com/scrapy-tutorial-web-scraping-with-python/?couponCode=SITE--SCRAPY-COUPON