目录

Hyperledger Fabric无排序组织以Raft协议启动多个Orderer服务、多组织共同运行维护Orderer服务

前言

Hyperledger Fabric无系统通道启动及通道的创建和删除中,我们已经完成了以无系统通道的方式启动 Hyperledger Fabric 网络,并将链码安装到指定通道。但目前为止,实验中的 orderer 服务都是通过单独的排序组织来维护且只有一个,那能不能排序排序组织而使用普通组织来运行维护多个 orderer 服务以降低网络复杂度呢?在全网检索后发现并没有人进行相关尝试,只能自己慢慢排坑了。本文将在之前的实验基础上,启动一个没有 orderer 组织的 Fabric 网络,网络中包含三个组织且每个组织运行维护着一个 Raft 协议的 orderer 节点,最后成功在其上部署运行链码。

工作准备

本文工作

以无排序组织的方式启动 Hyperledger Fabric 网络,其中包含四个组织—— council 、 soft 、 web 、 hard ,council 组织为网络提供 TLS-CA 服务,除了 council 外每个组织都运行维护着一个 peer 节点和一个 orderer,网络结构为(实验代码已上传至:https://github.com/wefantasy/FabricLearn4-1_RunOrdererByOneself 下):

运行端口 说明
council.ifantasy.net 7050 council 组织的 CA 服务, 为联盟链网络提供 TLS-CA 服务
soft.ifantasy.net 7250 soft 组织的 CA 服务, 包含成员: peer1 、 admin1
peer1.soft.ifantasy.net 7251 soft 组织的 peer1 成员节点
orderer1.soft.ifantasy.net 8251 soft 组织的 orderer1 服务
orderer1.soft.ifantasy.net 8252 soft 组织的 orderer1 服务的 admin 服务
web.ifantasy.net 7350 web 组织的 CA 服务, 包含成员: peer1 、 admin1
peer1.web.ifantasy.net 7351 web 组织的 peer1 成员节点
orderer1.soft.ifantasy.net 8351 web 组织的 orderer1 服务
orderer1.soft.ifantasy.net 8352 web 组织的 orderer1 服务的 admin 服务
hard.ifantasy.net 7450 hard 组织的 CA 服务, 包含成员: peer1 、 admin1
peer1.hard.ifantasy.net 7451 hard 组织的 peer1 成员节点
orderer1.soft.ifantasy.net 8451 hard 组织的 orderer1 服务
orderer1.soft.ifantasy.net 8452 hard 组织的 orderer1 服务的 admin 服务

实验准备

本文网络结构直接将 Hyperledger Fabric无系统通道启动及通道的创建和删除 中创建的 3_RunWithNoSystemChannel 复制为 4-1_RunOrdererByOneself (建议直接将本案例仓库 FabricLearn 下的 4-1_RunOrdererByOneself 目录拷贝到本地运行),文中大部分命令在 Hyperledger Fabric定制联盟链网络工程实践 中已有介绍因此不会详细说明。默认情况下,所有命令皆在 4-1_RunOrdererByOneself 根目录下执行,在开始后面的实验前按照以下命令启动基础实验网络(主要修改为删除 orderer 组织相关配置):

  1. 设置DNS(如果未设置): ./setDNS.sh
  2. 设置环境变量: source envpeer1soft
  3. 启动CA网络: ./0_Restart.sh

本实验初始 docker 网络为:

https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-09-24-43.png
初始 docker 网络

实验步骤

配置文件

  1. 修改配置文件 compose/docker-compose.yaml ,删除所有关于 orderer 组织的配置,并新增 hard 组织相关容器和普通组织的 orderer 容器:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
  hard.ifantasy.net:
    container_name: hard.ifantasy.net
    extends:
      file: docker-base.yaml
      service: ca-base
    command: sh -c 'fabric-ca-server start -d -b ca-admin:ca-adminpw --port 7050'
    environment:
      - FABRIC_CA_SERVER_CSR_CN=hard.ifantasy.net
      - FABRIC_CA_SERVER_CSR_HOSTS=hard.ifantasy.net
    volumes:
      - ${LOCAL_CA_PATH}/hard.ifantasy.net/ca:${DOCKER_CA_PATH}/ca
    ports:
      - 7450:7050
  peer1.hard.ifantasy.net:
    container_name: peer1.hard.ifantasy.net
    extends:
      file: docker-base.yaml
      service: peer-base
    environment:
      - CORE_PEER_ID=peer1.hard.ifantasy.net
      - CORE_PEER_LISTENADDRESS=0.0.0.0:7051
      - CORE_PEER_ADDRESS=peer1.hard.ifantasy.net:7051
      - CORE_PEER_LOCALMSPID=hardMSP
      - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.hard.ifantasy.net:7051
    volumes:
      - ${LOCAL_CA_PATH}/hard.ifantasy.net/registers/peer1:${DOCKER_CA_PATH}/peer
    ports:
      - 7451:7051

  orderer1.soft.ifantasy.net:
    container_name: orderer1.soft.ifantasy.net
    extends:
      file: docker-base.yaml
      service: orderer-base
    environment:
      - ORDERER_HOST=orderer1.soft.ifantasy.net
      - ORDERER_GENERAL_LOCALMSPID=softMSP
      - ORDERER_GENERAL_LISTENPORT=8251
    volumes:
      - ${LOCAL_CA_PATH}/soft.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer
    ports:
      - 8251:8251
      - 8252:8888
      - 8253:9999
      
  orderer1.web.ifantasy.net:
    container_name: orderer1.web.ifantasy.net
    extends:
      file: docker-base.yaml
      service: orderer-base
    environment:
      - ORDERER_HOST=orderer1.web.ifantasy.net
      - ORDERER_GENERAL_LOCALMSPID=webMSP
      - ORDERER_GENERAL_LISTENPORT=8351
    volumes:
      - ${LOCAL_CA_PATH}/web.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer
    ports:
      - 8351:8351
      - 8352:8888
      - 8353:9999

  orderer1.hard.ifantasy.net:
    container_name: orderer1.hard.ifantasy.net
    extends:
      file: docker-base.yaml
      service: orderer-base
    environment:
      - ORDERER_HOST=orderer1.hard.ifantasy.net
      - ORDERER_GENERAL_LOCALMSPID=hardMSP
      - ORDERER_GENERAL_LISTENPORT=8451
    volumes:
      - ${LOCAL_CA_PATH}/hard.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer
    ports:
      - 8451:8451
      - 8452:8888
      - 8453:9999
  1. 修改配置文件 config/configtx.yaml源文件太长在此不贴,其主要修改内容为:

    • 每个组织 MSP 下增加本组织维护的 OrdererEndpoints 配置:
      https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-09-42-29.png
      增加本组织维护的 OrdererEndpoints
    • Orderer 配置下修改 orderer 服务的地址:
      https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-09-43-32.png
      Orderer 配置
    • Profiles 配置下修改排序节点的维护组织为 softMSPwebMSPhardMSP
      https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-09-44-40.png
      Profiles 配置
    • 必须有一个组织 MSP 的 Policies 中的 ReadersWritersRule 值为 member ,文末会有解释:
      https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-15-15-46.png
      组织 Policies
  2. 各组织的环境变量文件中添加 orderer 服务的管理证书环境变量,以 envpeer1soft 为例:

1
2
3
export ORDERER_CA=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/tlscacerts/tls-council-ifantasy-net-7050.pem
export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/signcerts/cert.pem
export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
  1. envpeer1soft 复制为 envpeer1hard 作为 hard 组织的环境变量,其内容为:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
export LOCAL_ROOT_PATH=$PWD
export LOCAL_CA_PATH=$LOCAL_ROOT_PATH/orgs
export DOCKER_CA_PATH=/tmp
export COMPOSE_PROJECT_NAME=fabriclearn
export DOCKER_NETWORKS=network
export FABRIC_BASE_VERSION=2.4
export FABRIC_CA_VERSION=1.5
echo "init terminal hard"
export FABRIC_CFG_PATH=$LOCAL_ROOT_PATH/config
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="hardMSP"
export CORE_PEER_ADDRESS=peer1.hard.ifantasy.net:7451
export CORE_PEER_TLS_ROOTCERT_FILE=$LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
export CORE_PEER_MSPCONFIGPATH=$LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp

export ORDERER_CA=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/tlscacerts/tls-council-ifantasy-net-7050.pem
export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/signcerts/cert.pem
export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem

注册用户

直接运行根目录下的 1_RegisterUser.sh 即可完成本实验所需用户的注册。

  1. council 用户注册:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
echo "Working on council"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:[email protected]:7050
fabric-ca-client register -d --id.name peer1soft --id.secret peer1soft --id.type peer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer1soft --id.secret orderer1soft --id.type orderer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name peer1web --id.secret peer1web --id.type peer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer1web --id.secret orderer1web --id.type orderer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name peer1hard --id.secret peer1hard --id.type peer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer1hard --id.secret orderer1hard --id.type orderer -u https://council.ifantasy.net:7050
  1. soft 用户注册:
1
2
3
4
5
6
7
echo "Working on soft"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:[email protected]:7250
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://soft.ifantasy.net:7250
fabric-ca-client register -d --id.name orderer1 --id.secret orderer1 --id.type orderer -u https://soft.ifantasy.net:7250
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://soft.ifantasy.net:7250
  1. web 用户注册:
1
2
3
4
5
6
7
echo "Working on web"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:[email protected]:7350
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://web.ifantasy.net:7350
fabric-ca-client register -d --id.name orderer1 --id.secret orderer1 --id.type orderer -u https://web.ifantasy.net:7350
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://web.ifantasy.net:7350
  1. hard 用户注册:
1
2
3
4
5
6
7
8
echo "Working on hard"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:[email protected]:7450
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://hard.ifantasy.net:7450
fabric-ca-client register -d --id.name orderer1 --id.secret orderer1 --id.type orderer -u https://hard.ifantasy.net:7450
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://hard.ifantasy.net:7450
echo "All CA and registration done"

组织证书构建

直接运行根目录下的 2_EnrollUser.sh 即可完成本实验所需证书的构建。

  1. 组织资产预处理:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
echo "Preparation============================="
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/assets
cp $LOCAL_CA_PATH/soft.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem

mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/assets 
cp $LOCAL_CA_PATH/web.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem

mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/assets
cp $LOCAL_CA_PATH/hard.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
echo "Preparation============================="
  1. soft 证书构建:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
echo "Start Soft============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:[email protected]:7250
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:[email protected]:7250
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1soft:[email protected]:7050 --enrollment.profile tls --csr.hosts peer1.soft.ifantasy.net
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

echo "Enroll Orderer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer1:[email protected]:7250
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1soft:[email protected]:7050 --enrollment.profile tls --csr.hosts orderer1.soft.ifantasy.net
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/msp/admincerts/cert.pem

mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/soft.ifantasy.net/msp/config.yaml
echo "End Soft============================="
  1. web 证书构建:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
echo "Start Web============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:[email protected]:7350
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:[email protected]:7350
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1web:[email protected]:7050 --enrollment.profile tls --csr.hosts peer1.web.ifantasy.net
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

echo "Enroll Orderer1"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/registers/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer1:[email protected]:7350
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/registers/orderer1/msp/admincerts
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/registers/orderer1/msp/admincerts/cert.pem
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1web:[email protected]:7050 --enrollment.profile tls --csr.hosts orderer1.web.ifantasy.net
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/orderer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/web.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem

mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/web.ifantasy.net/msp/config.yaml
echo "End Web============================="
  1. hard 证书构建:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
echo "Start Hard============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:[email protected]:7450
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:[email protected]:7450
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1hard:[email protected]:7050 --enrollment.profile tls --csr.hosts peer1.hard.ifantasy.net
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

echo "Enroll Orderer1"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer1:[email protected]:7450
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/msp/admincerts
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/msp/admincerts/cert.pem
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1hard:[email protected]:7050 --enrollment.profile tls --csr.hosts orderer1.hard.ifantasy.net
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem

mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/hard.ifantasy.net/msp/config.yaml
echo "End Hard============================="

配置通道

直接运行根目录下的 3_Configtxgen.sh 即可完成本实验所需通道配置。

  1. 启动 orderer 和 peer 容器:
1
2
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d peer1.soft.ifantasy.net peer1.web.ifantasy.net peer1.hard.ifantasy.net 
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d orderer1.soft.ifantasy.net orderer1.web.ifantasy.net orderer1.hard.ifantasy.net

此时已启动所有实验所需容器:

https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-11-14-02.png
实验所需容器
2. 创建创世区块:

1
2
3
4
5
6
configtxgen -profile OrgsChannel -outputCreateChannelTx $LOCAL_ROOT_PATH/data/testchannel.tx -channelID testchannel
configtxgen -profile OrgsChannel -outputBlock $LOCAL_ROOT_PATH/data/testchannel.block -channelID testchannel

cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/soft.ifantasy.net/assets/
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/web.ifantasy.net/assets/
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/hard.ifantasy.net/assets/
  1. 使三个 orderer 加入 testchannel 通道:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
source envpeer1soft
osnadmin channel list -o orderer1.soft.ifantasy.net:8252 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
osnadmin channel join -o orderer1.soft.ifantasy.net:8252 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer1.soft.ifantasy.net:8252 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
source envpeer1web
osnadmin channel list -o orderer1.web.ifantasy.net:8352 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
osnadmin channel join -o orderer1.web.ifantasy.net:8352 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer1.web.ifantasy.net:8352 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
source envpeer1hard
osnadmin channel list -o orderer1.hard.ifantasy.net:8452 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
osnadmin channel join -o orderer1.hard.ifantasy.net:8452 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer1.hard.ifantasy.net:8452 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
  1. 三组织加入 testchannel 通道:
1
2
3
4
5
6
7
8
9
source envpeer1soft
peer channel join -b $LOCAL_CA_PATH/soft.ifantasy.net/assets/testchannel.block
peer channel list
source envpeer1web
peer channel join -b $LOCAL_CA_PATH/web.ifantasy.net/assets/testchannel.block
peer channel list
source envpeer1hard
peer channel join -b $LOCAL_CA_PATH/hard.ifantasy.net/assets/testchannel.block
peer channel list

部署测试链码

直接运行根目录下的 4_TestChaincode.sh 即可完成本实验示例的链码安装和测试。

  1. 各组织安装测试链码:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
source envpeer1soft
# peer lifecycle chaincode package basic.tar.gz --path asset-transfer-basic/chaincode-go --label basic_1
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled
source envpeer1web
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled
source envpeer1hard
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled
  1. 设置链码 ID 环境变量:
1
export CHAINCODE_ID=basic_1:06613e463ef6694805dd896ca79634a2de36fdf019fa7976467e6e632104d718
  1. soft 组织批准链码:
1
2
3
source envpeer1soft
peer lifecycle chaincode approveformyorg -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
  1. web 组织批准链码:
1
2
3
source envpeer1web
peer lifecycle chaincode approveformyorg -o orderer1.web.ifantasy.net:8351 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
  1. hard 组织批准链码:
1
2
3
source envpeer1hard
peer lifecycle chaincode approveformyorg -o orderer1.hard.ifantasy.net:8451 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1

注意,这里各组织批准链码时的 -o 参数可以指定任意一个 orderer 服务

https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-11-02-34.png
批准链码
4. 检查链码批准情况:

1
peer lifecycle chaincode checkcommitreadiness -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --init-required
  1. 测试调用链码:
1
2
3
4
5
6
source envpeer1soft
peer lifecycle chaincode commit -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --init-required --version 1.0 --sequence 1 --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE
peer lifecycle chaincode querycommitted --channelID testchannel --name basic -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE
peer chaincode invoke --isInit -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["InitLedger"]}'
sleep 3
peer chaincode invoke -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["GetAllAssets"]}'

https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-10-25-52.png
测试调用链码:

常见错误

  1. 没有领导节点
1
Error: failed to send transaction: got unexpected status: SERVICE_UNAVAILABLE -- no Raft leader

上述错误归结起来就是 orderer 之间没有选出领导节点,此时应该检查:

  • 网络中 orderer 节点的数量是否为 2n+1 个,否则可能无法完成选举
  • 各 orderer 容器的 ORDERER_GENERAL_LOCALMSPID 配置是否正确,必须为自身所属组织的 MSPID
  • 检查 configtx.yaml 中各组织的 Policies 配置是否正确
  1. 排序节点之间无法通信
1
2022-04-09 05:32:07.086 UTC 0032 ERRO [orderer.consensus.etcdraft] logSendFailure -> Failed to send StepRequest to 3, because: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 172.19.0.10:8451: connect: connection refused" channel=syschannel node=1

上述错误的原因是 orderer 节点间无法通信, 此时应该检查 configtx.yaml 中相关的 orderer 地址是否正确。这里有个大坑:所有 configtx.yaml 文件内的 orderer 相关配置的端口必须设置为容器内 ORDERER_GENERAL_LISTENPORT 的监听端口,而不是容器外的映射端口,假如 orderer 容器配置如下图, configtx.yaml 中的 orderer 端口必须为 7050 而不能填 8251 (所以为了避免冲突,强烈建议这两个端口设置成一样的 8251)。

https://cdn.jsdelivr.net/gh/wefantasy/FileCloud/img/hyperledger_fabric_5_run_orderer_by_oneself-2022-04-12-10-35-56.png
orderer 示例配置

  1. peer 节点之间无法通信
1
2
Error: timed out waiting for txid on all peers
2022-04-10 02:57:37.135 UTC 00a1 WARN [peer.blocksprovider] DeliverBlocks -> Got error while attempting to receive blocks: block from orderer could not be verified: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Writers' sub-policies to be satisfied channel=testchannel orderer-address=orderer1.soft.ifantasy.net:8251

错误原因是没有操作权限,通常是 configtx.yaml 中的策略问题,在本实验中如果三个组织的 Policies 都设置为下列内容则会触发本错误:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Policies:
    Readers:
        Type: Signature
        Rule: "OR('softMSP.admin', 'softMSP.peer', 'softMSP.client')"
    Writers:
        Type: Signature
        Rule: "OR('softMSP.admin', 'softMSP.client')"
    Admins:
        Type: Signature
        Rule: "OR('softMSP.admin')"
    Endorsement:
        Type: Signature
        Rule: "OR('softMSP.peer')"

此时需要将任意组织(比如 web)的 ReadersWritersRule 改为 menber 即可解决,解决后实验各步骤结果符合预期:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Policies:
    Readers:
        Type: Signature
        Rule: "OR('webMSP.member')"
    Writers:
        Type: Signature
        Rule: "OR('webMSP.member')"
    Admins:
        Type: Signature
        Rule: "OR('webMSP.admin')"
    Endorsement:
        Type: Signature
        Rule: "OR('webMSP.peer')"

至于为什么会导致如此尚未发现,猜测是普通组织的策略与排序节点所需要的策略存在冲突,因此建议排序服务独立于普通组织