windows下PyCharm运行和调试scrapy

Creating a project

1. cmd 开启虚拟环境 env\Scripts activate

scrapy startproject tutorial

This will create a tutorial directory with the following contents:


tutorial/
    scrapy.cfg            # deploy configuration file

    tutorial/             # project's Python module, you'll import your code from here
        __init__.py

        items.py          # project items definition file

        pipelines.py      # project pipelines file

        settings.py       # project settings file

        spiders/          # a directory where you'll later put your spiders
            __init__.py

This is the code for our first Spider. Save it in a file named quotes_spider.py under thetutorial/spiders directory in your project:

Scrapy是爬虫抓取框架，Pycharm是强大的python的IDE，为了方便使用需要在PyCharm对scrapy程序进行调试 python PyCharm Scrapy 

scrapy指令其实就是一个python的运行脚本

在运行scrapy库时，其实是相当于运行一个python脚本

所以当运行scrapy crawler spider时，其实与指令的效果是一样的
python /Library/Python/2.7/site-packages/scrapy/cmdline.py crawl spider

这样只需要在pycharm中的Run下Edit Configurations...中做运行配置即可：