Usage: nutch COMMAND

where COMMAND is one of:

  crawl             one-step crawler for intranets (DEPRECATED - USE CRAWL SCRIPT INSTEAD)

  readdb            read / dump crawl db

  mergedb           merge crawldb-s, with optional filtering

  readlinkdb        read / dump link db

  inject            inject new urls into the database

  generate          generate new segments to fetch from crawl db

  freegen           generate new segments to fetch from text files

  fetch             fetch a segment's pages

  parse             parse a segment's pages

  readseg           read / dump segment data

  mergesegs         merge several segments, with optional filtering and slicing

  updatedb          update crawl db from segments after fetching

  invertlinks       create a linkdb from parsed segments

  mergelinkdb       merge linkdb-s, with optional filtering

  index             run the plugin-based indexer on parsed segments and linkdb

  solrindex         run the solr indexer on parsed segments and linkdb

  solrdedup         remove duplicates from solr

  solrclean         remove HTTP 301 and 404 documents from solr

  clean             remove HTTP 301 and 404 documents from indexing backends configured via plugins

  parsechecker      check the parser for a given url

  indexchecker      check the indexing filters for a given url

  domainstats       calculate domain statistics from crawldb

  webgraph          generate a web graph from existing segments

  linkrank          run a link analysis program on the generated web graph

  scoreupdater      updates the crawldb with linkrank scores

  nodedumper        dumps the web graph's node scores

  plugin            load a plugin and run one of its classes main()

  junit             runs the given JUnit test

or

  CLASSNAME         run the class named CLASSNAME

Most commands print help when invoked w/o parameters.

新建urls目录，用于存放首要抓取的url列表

?

1 2	[hadoop@localhostusr]# mkdir urls [hadoop@localhostusr]# vim urls/seed.txt

写入 

http://www.taobao.com/

如果多个的话每一行写一个url

修改抓取url正则，仅允许抓

taobao.com上的内容。

?

1	[hadoop@localhostusr]# vim conf/regex-urlfilter.txt

 修改

?

1 2	#accept anything else +.

为

?

1	+^http://([a-z0-9]*\.)*taobao.com/

如果不限制的话就不需要修改