概述
----
通常抓取级联数数据情况不多，但要是真需要时，确多了一些麻烦，比如抓取商品分类级别信息等。本内容将讲述如何采集无限级联联数据，并以GoldData来抓取2019年最新的省市县三级为示例。

创建数据集
----

在数据集管理里，添加数据集area。如下图所示：

![dataset](https://oscimg.oschina.net/oscnet/0d30f37ec0330df06390c72f699463ce86c.jpg "dataset")

数据集相当于数据库中的表，只是字段是灵活的，可以随着需要而添加和变更。

创建规则
----
在规则管理里，添加规则arearule，并将地址http://xzqh.mca.gov.cn/map 填为抓取入口地址。

我们通过分析可知，我们可从http://xzqh.mca.gov.cn/map 获取省级数据，然后通过级级数据请求http://xzqh.mca.gov.cn/selectJson 带有省级名称来获得市级数据，最后再通过市级数据分别请求http://xzqh.mca.gov.cn/selectJson 带有市级数据来获得县级数据。

并且发现请求 http://xzqh.mca.gov.cn/selectJson 是需要发POST请求的，因此我们需要将url加个前缀fake:,然后用规则里用JavaScript去请求URL。

我们在此编写的area数据集，有以下字段：

| 名称 |说明|
|------|----|
| sn   | 取区域编码作为记录唯一字段|
|name  |名称|
|code |取区域编码|
|abbr |省名简写|
| parent_code| 取父区域编码|

因此编写规则如下：

![rule](https://oscimg.oschina.net/oscnet/08c01df3c6191ba285260edce9595ba1e88.jpg "rule")

（注：具体规则内容请见文章最后）

然后编写完成后，我们就可以启动抓取器进行抓取。

查看数据
------
打开数据管理，选择area数据集查看如下图所示：

![area](https://oscimg.oschina.net/oscnet/abeba8e26a71f0b97bbf823d8465ff27bc8.jpg "area")

导出数据
--------

回到数据管理，选择条件，选择需要导出的字段，进行导出。像这里数据比较多，GoldData将会以打包excel文件并压缩为zip文件下载。解压到本地，然后打开excel就可以看到抓取的数据，如下图所示：

![area_export](https://oscimg.oschina.net/oscnet/35e3edeef292507135530a593169d3ad475.jpg "area_export")

结尾
------

通过这节内容，可以了解了如何通过GoldData 抓取级联数据。但是下一个问题是如何将数据导入自关联列表呢，且看下一往篇将会讲述如何将级联数据融合到数据库自关联表当中。

咐录：
-----
**（抓取规则）**

```json
[
  {
    __sample: http://xzqh.mca.gov.cn/map
    match0: http\:\/\/xzqh\.mca\.gov\.cn\/map
    fields0:
    {
      __model: true
      __node: js
      __js:
        '''
        var exp11=/json\s=\s(.+)\s+\$$doc/
        var ret=exp11.exec(html)
        var ss=eval(ret[1])
        for(var i=0;i<ss.length;i++){
           var ele=ss[i]
           var exp12=/\((.+)$/.exec(ele.shengji)
        var exp13=/(.+)\(/.exec(ele.shengji)
           out.add({sn:ele.quHuaDaiMa,name:exp13[1],code:ele.quHuaDaiMa,abbr:exp12[1],
         parent_code:null,

})
        }

'''
      name:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      abbr:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      sn:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      __dataset: area
      parent_code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
    }
    fields1:
    {
      __node: js
      __js:
        '''
        var exp11=/json\s=\s(.+)\s+\$\(doc/
        var ret=exp11.exec(html)
        var ss=eval(ret[1])
        for(var i=0;i<ss.length;i++){
           var ele=ss[i]

var url='fake:http://xzqh.mca.gov.cn/selectJson?shengji='+ele.shengji+"&code="+ele.quHuaDaiMa
           //var $ajax(url,[__method:'POST',data:'shengji='+ele.shengji]).content
           
           out.add({href:url})
        }

'''
      href:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
    }
  }
  {
    __sample: fake:http://xzqh.mca.gov.cn/selectJson?shengji=北京市(京)&code=110000
    match0: fake\:http\:\/\/xzqh\.mca\.gov\.cn\/selectJson\?shengji=.+\&code=\d+
    fields0:
    {
      __model: true
      __dataset: area
      sn:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      __node: js
      __js:
        '''
        var urlRet= /http\:\/\/xzqh\.mca\.gov\.cn\/selectJson\?shengji=(.+)\&code=(\d+)/.exec(baseUri);
        var shengji=urlRet[1]
        var sjCode=urlRet[2]

var url='http://xzqh.mca.gov.cn/selectJson'
        var  content=$ajax(url,{__method:'POST',data:'shengji='+shengji }).content

var arr=eval(content);
        for(var i=0;i<arr.length;i++){
           var dj=arr[i];
           
            var area={
                 name:dj.diji,
                 code:dj.quHuaDaiMa,
                 parent_code:sjCode,
                 sn:dj.quHuaDaiMa,
            }
           out.add(area);
        }

'''
      name:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      abbr:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      sn:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      parent_code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
    }
    fields1:
    {
      __node: js
      __js:
        '''
        var urlRet= /http\:\/\/xzqh\.mca\.gov\.cn\/selectJson\?shengji=(.+)\&code=(\d+)/.exec(baseUri);
        var shengji=urlRet[1]
        var sjCode=urlRet[2]

var url='http://xzqh.mca.gov.cn/selectJson'
        var  content=$ajax(url,{__method:'POST',data:'shengji='+shengji }).content

var arr=eval(content);
        for(var i=0;i<arr.length;i++){
           var dj=arr[i];

var url='fake:http://xzqh.mca.gov.cn/selectJson?shengji='+shengji+"&sjcode="+sjCode+"&diji="+dj.diji+"&djcode="+dj.quHuaDaiMa   
           out.add({href:url})
        }

'''
      href:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
    }
  }
  {
    __sample: fake:href :fake:http://xzqh.mca.gov.cn/selectJson?shengji=北京市(京)&sjcode=110000&diji=北京市&djcode=110000
    match0: fake\:http\:\/\/xzqh\.mca\.gov\.cn/selectJson\?shengji=([^&]+)\&sjcode=([^&]+)\&diji=([^&]+)\&djcode=([^&]+)
    fields0:
    {
      __model: true
      __dataset: area
      __node: js
      sn:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      __js:
        '''
        var urlRet=/fake\:http\:\/\/xzqh\.mca\.gov\.cn\/selectJson\?shengji=([^\&]+)\&sjcode=([^\&]+)\&diji=([^\&]+)\&djcode=([^\&]+)/.exec(baseUri)

var shengji=urlRet[1]
        var sjCode=urlRet[2]
        var dj=urlRet[3]
        var djCode=urlRet[4]

var url='http://xzqh.mca.gov.cn/selectJson'
        var  content=$ajax(url,{__method:'POST',data:'shengji='+shengji+'&diji='+dj }).content

var arr=eval(content);
        for(var i=0;i<arr.length;i++){
           var dj=arr[i];
           
            var area={
                 name:dj.xianji,
                 code:dj.quHuaDaiMa,
                 parent_code:djCode,
                 sn:dj.quHuaDaiMa,
            }
           out.add(area);
        }

'''
      name:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      abbr:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      sn:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
      parent_code:
      {
        expr: ""
        attr: ""
        js: ""
        __label: ""
        __showOnList: false
        __type: ""
        down: "0"
        accessPathJs: ""
        uploadConf: ""
      }
    }
  }
]
```

ITPub博客

如何采集级联数据(比如最新省市县)呢？

最新文章