一些问题...

[evi-Genius]

您好，我正在阅读并测试您的代码，我有一些疑惑和问题想要请教您。
我想复现white paper里mobileNetv2的per-channel量化，结果在imageNet val上面测试结果只有0.09%，说明per-channel的方案完全没有效果，请问您有什么头绪吗？

[evi-Genius]

************************* Settings *************************
eval_per_calib : False
use_gpu : 2
batch_norm : False
use_gn : False
input_bits_width : 8
exclude_first_conv : true
dataset : imagenet
batch_size : 64
use_se : False
print_model : False
input_signed : True
list_models : False
disable_cudnn_autotune : False
last_gamma : False
num_workers : 6
num_sample : 5
calib_mode : kl
quantize_input_offline : True
fixed_random_seed : 7
merge_bn : True
calib_epoch : 3
model : mobilenetv2_1.0
quant_type : channel
weight_bits_width : 8

---

[evi-Genius]

会不会是因为weight量化方案采用max对称量化的关系？我看depth wise convulation上面权重的异常值比较多，导致max对称量化后信息损失太大？

[hey-yahei]

不好意思，时隔一个多月今天才看到这个issue。经过排查发现是input的量化有问题，先前都只考虑了relu+conv的情况，没有考虑负值inputs，现在已经更新了代码。但是，

1. 目前还无法为mbv2使用KL校准，主要原因是np.bincount不支持负值，目前还没来得及解决这个问题，后续会处理一下；
2. mbv2在在线量化的表现还行，但naive校准的离线量化效果不佳，可能是input的数值分布差异比较大导致的。

---

$ python simulate_quantization.py --model=mobilenetv2_1.0 --use-gpu=1 --merge-bn --quant-type=channel --input-signed=true

************************* Settings *************************
model : mobilenetv2_1.0
print_model : False
list_models : False
use_gpu : 1
dataset : imagenet
use_gn : False
batch_norm : False
use_se : False
last_gamma : False
merge_bn : True
weight_bits_width : 8
input_signed : true
input_bits_width : 8
quant_type : channel
num_workers : 4
batch_size : 128
num_sample : 5
quantize_input_offline : False
calib_mode : naive
calib_epoch : 3
disable_cudnn_autotune : False
eval_per_calib : False
exclude_first_conv : true
fixed_random_seed : 7

---

************************* Exclude blocks *************************
mobilenetv20_features_conv0
mobilenetv20_features_batchnorm0
mobilenetv20_output_pred

---

Eval: 0%| | 0/391 [00:00<?, ?it/s][15:30:48] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Eval: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌| 390/391 [03:02<00:00, 2.60it/s][15:33:52] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Eval: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 391/391 [03:05<00:00, 1.21s/it]
************************* Result *************************
acc : 71.99%
avg_acc : 71.99%

---

---

$ python simulate_quantization.py --model=mobilenetv2_1.0 --use-gpu=1 --merge-bn --quant-type=channel --input-signed=true --quantize-input-offline

************************* Settings *************************
model : mobilenetv2_1.0
print_model : False
list_models : False
use_gpu : 1
dataset : imagenet
use_gn : False
batch_norm : False
use_se : False
last_gamma : False
merge_bn : True
weight_bits_width : 8
input_signed : true
input_bits_width : 8
quant_type : channel
num_workers : 4
batch_size : 128
num_sample : 5
quantize_input_offline : True
calib_mode : naive
calib_epoch : 3
disable_cudnn_autotune : False
eval_per_calib : False
exclude_first_conv : true
fixed_random_seed : 7

---

************************* Exclude blocks *************************
mobilenetv20_features_conv0
mobilenetv20_features_batchnorm0
mobilenetv20_output_pred

---

************************* Naive Calibration *************************
Calib[1/3]: 0%| | 0/40 [00:00<?, ?it/s][15:45:32] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Calib[1/3]: 98%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 39/40 [00:21<00:00, 2.46it/s][15:45:53] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Calib[1/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:23<00:00, 1.28it/s]
Calib[2/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:18<00:00, 3.05it/s]
Calib[3/3]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:18<00:00, 3.12it/s]
Best threshold for mobilenetv20_features_linearbottleneck0_conv0: 5.988860607147217
Best threshold for mobilenetv20_features_linearbottleneck0_conv1: 5.999978542327881
Best threshold for mobilenetv20_features_linearbottleneck1_conv0: 18.790611267089844
Best threshold for mobilenetv20_features_linearbottleneck1_conv1: 5.999978542327881
Best threshold for mobilenetv20_features_linearbottleneck1_conv2: 5.984673023223877
Best threshold for mobilenetv20_features_linearbottleneck2_conv0: 13.01661205291748
Best threshold for mobilenetv20_features_linearbottleneck2_conv1: 5.990994453430176
Best threshold for mobilenetv20_features_linearbottleneck2_conv2: 5.999978542327881
Best threshold for mobilenetv20_features_linearbottleneck3_conv0: 18.15996742248535
Best threshold for mobilenetv20_features_linearbottleneck3_conv1: 5.973598957061768
Best threshold for mobilenetv20_features_linearbottleneck3_conv2: 5.948035717010498
Best threshold for mobilenetv20_features_linearbottleneck4_conv0: 12.318225860595703
Best threshold for mobilenetv20_features_linearbottleneck4_conv1: 5.672513961791992
Best threshold for mobilenetv20_features_linearbottleneck4_conv2: 5.92328405380249
Best threshold for mobilenetv20_features_linearbottleneck5_conv0: 14.465754508972168
Best threshold for mobilenetv20_features_linearbottleneck5_conv1: 5.8163628578186035
Best threshold for mobilenetv20_features_linearbottleneck5_conv2: 5.846293926239014
Best threshold for mobilenetv20_features_linearbottleneck6_conv0: 17.13546371459961
Best threshold for mobilenetv20_features_linearbottleneck6_conv1: 5.877142906188965
Best threshold for mobilenetv20_features_linearbottleneck6_conv2: 5.84243106842041
Best threshold for mobilenetv20_features_linearbottleneck7_conv0: 9.198147773742676
Best threshold for mobilenetv20_features_linearbottleneck7_conv1: 4.9297709465026855
Best threshold for mobilenetv20_features_linearbottleneck7_conv2: 5.560483932495117
Best threshold for mobilenetv20_features_linearbottleneck8_conv0: 10.592511177062988
Best threshold for mobilenetv20_features_linearbottleneck8_conv1: 5.073105335235596
Best threshold for mobilenetv20_features_linearbottleneck8_conv2: 5.82119083404541
Best threshold for mobilenetv20_features_linearbottleneck9_conv0: 11.359800338745117
Best threshold for mobilenetv20_features_linearbottleneck9_conv1: 5.044006824493408
Best threshold for mobilenetv20_features_linearbottleneck9_conv2: 5.929920196533203
Best threshold for mobilenetv20_features_linearbottleneck10_conv0: 11.696471214294434
Best threshold for mobilenetv20_features_linearbottleneck10_conv1: 5.494167804718018
Best threshold for mobilenetv20_features_linearbottleneck10_conv2: 5.944950103759766
Best threshold for mobilenetv20_features_linearbottleneck11_conv0: 9.514467239379883
Best threshold for mobilenetv20_features_linearbottleneck11_conv1: 5.645473957061768
Best threshold for mobilenetv20_features_linearbottleneck11_conv2: 5.95599889755249
Best threshold for mobilenetv20_features_linearbottleneck12_conv0: 10.617852210998535
Best threshold for mobilenetv20_features_linearbottleneck12_conv1: 5.6360182762146
Best threshold for mobilenetv20_features_linearbottleneck12_conv2: 5.9353556632995605
Best threshold for mobilenetv20_features_linearbottleneck13_conv0: 12.687278747558594
Best threshold for mobilenetv20_features_linearbottleneck13_conv1: 5.4570746421813965
Best threshold for mobilenetv20_features_linearbottleneck13_conv2: 5.798008441925049
Best threshold for mobilenetv20_features_linearbottleneck14_conv0: 7.8866963386535645
Best threshold for mobilenetv20_features_linearbottleneck14_conv1: 5.743549823760986
Best threshold for mobilenetv20_features_linearbottleneck14_conv2: 5.802321434020996
Best threshold for mobilenetv20_features_linearbottleneck15_conv0: 9.552735328674316
Best threshold for mobilenetv20_features_linearbottleneck15_conv1: 5.421483516693115
Best threshold for mobilenetv20_features_linearbottleneck15_conv2: 5.760587215423584
Best threshold for mobilenetv20_features_linearbottleneck16_conv0: 11.939920425415039
Best threshold for mobilenetv20_features_linearbottleneck16_conv1: 5.075140953063965
Best threshold for mobilenetv20_features_linearbottleneck16_conv2: 5.953917503356934
Best threshold for mobilenetv20_features_conv1: 5.507375240325928

---

Eval: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 391/391 [03:05<00:00, 1.21s/it]
************************* Result *************************
acc : 65.17%
avg_acc : 65.17%

---

[evi-Genius]

明白了，感谢回复！