昇腾910使用记录

news/2024/6/3 17:32:47 标签: 人工智能, linux, 运维, 华为, 云计算

一. 压缩文件和解压文件

1. 压缩文件

tar -czvf UNITE-main.tar.gz ./UNITE-main/

2. 解压文件

tar -xvf ./UNITE-main/

二. CUDA更改为NPU

data['label'] = data['label'].cuda()
data['instance'] = data['instance'].cuda()
data['image'] = data['image'].cuda()

更改为

data['label'] = data['label'].npu()
data['instance'] = data['instance'].npu()
data['image'] = data['image'].npu()

三. 配置环境变量

1. 创建env.sh

touch env.sh

2. 打开env.sh

vi env.sh

3. 配置环境变量

# 配置CANN相关环境变量
CANN_INSTALL_PATH_CONF='/etc/Ascend/ascend_cann_install.info'
if [ -f $CANN_INSTALL_PATH_CONF ]; then
 DEFAULT_CANN_INSTALL_PATH=$(cat $CANN_INSTALL_PATH_CONF | grep Install_Path | cut -d "=" -f 2)
else
 DEFAULT_CANN_INSTALL_PATH="/usr/local/Ascend/"
fi
CANN_INSTALL_PATH=${1:-${DEFAULT_CANN_INSTALL_PATH}}
if [ -d ${CANN_INSTALL_PATH}/ascend-toolkit/latest ];then
 source ${CANN_INSTALL_PATH}/ascend-toolkit/set_env.sh
else
 source ${CANN_INSTALL_PATH}/nnae/set_env.sh
fi
# 导入依赖库
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/openblas/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib/aarch64_64-linux-gnu
# 配置自定义环境变量
export HCCL_WHITELIST_DISABLE=1
# log
export ASCEND_SLOG_PRINT_TO_STDOUT=0 # 日志打屏, 可选
export ASCEND_GLOBAL_LOG_LEVEL=3 # 日志级别常用 1 INFO级别; 3 ERROR级别
export ASCEND_GLOBAL_EVENT_ENABLE=0 # 默认不使能event日志信息

并输入

:wq!

4. 使用环境

source env.sh

四. RuntimeError: ACL stream synchronize failed, error code:507018

E39999: Inner Error, Please contact support engineer!
E39999  Aicpu kernel execute failed, device_id=0, stream_id=0, task_id=6394, fault op_name=ScatterElements[FUNC:GetError][FILE:stream.cc][LINE:1044]
        TraceBack (most recent call last):
        rtStreamSynchronize execute failed, reason=[aicpu exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:49]
        synchronize stream failed, runtime result = 507018[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]


DEVICE[0] PID[41411]: 
EXCEPTION TASK:
  Exception info:TGID=2593324, model id=65535, stream id=0, stream phase=SCHEDULE, task id=742, task type=aicpu kernel, recently received task id=742, recently send task id=741, task phase=RUN
  Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
    Other info[0]:time=2023-10-12-11:22:01.273.951, function=proc_aicpu_task_done, line=972, error code=0x2a 
EXCEPTION TASK:
  Exception info:TGID=2593324, model id=65535, stream id=0, stream phase=3, task id=6394, task type=aicpu kernel, recently received task id=6406, recently send task id=6393, task phase=RUN
  Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
    Other info[0]:time=2023-10-12-11:41:20.661.958, function=proc_aicpu_task_done, line=972, error code=0x2a
Traceback (most recent call last):
  File "train.py", line 40, in <module>
    trainer.run_generator_one_step(data_i)
  File "/home/ma-user/work/SPADE-master/trainers/pix2pix_trainer.py", line 35, in run_generator_one_step
    g_losses, generated = self.pix2pix_model(data, mode='generator')
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
    return self.module(*inputs, **kwargs)
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ma-user/work/SPADE-master/models/pix2pix_model.py", line 43, in forward
    input_semantics, real_image = self.preprocess_input(data)
  File "/home/ma-user/work/SPADE-master/models/pix2pix_model.py", line 113, in preprocess_input
    data['label'] = data['label'].npu()
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/site-packages/torch_npu/utils/device_guard.py", line 38, in wrapper
    return func(*args, **kwargs)
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/site-packages/torch_npu/utils/tensor_methods.py", line 66, in _npu
    return torch_npu._C.npu(self, *args, **kwargs)
RuntimeError: ACL stream synchronize failed, error code:507018
THPModule_npu_shutdown success.

猜测可能是没有开混合精度

五. 开启混合精度

1. 在构建神经网络前，我们需要导入torch_npu中的AMP模块

import time
import torch
import torch.nn as nn
import torch_npu
from torch_npu.npu import amp    # 导入AMP模块

2. 在模型、优化器定义之后，定义AMP功能中的GradScaler

model = CNN().to(device)
train_dataloader = DataLoader(train_data, batch_size=batch_size)    # 定义DataLoader
loss_func = nn.CrossEntropyLoss().to(device)    # 定义损失函数
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)    # 定义优化器
scaler = amp.GradScaler()    # 在模型、优化器定义之后，定义GradScaler

3. 在训练代码中添加AMP功能相关的代码开启AMP

for epo in range(epochs):
for imgs, labels in train_dataloader:
imgs = imgs.to(device)
    labels = labels.to(device)
    with amp.autocast():
        outputs = model(imgs)    # 前向计算
        loss = loss_func(outputs, labels)    # 损失函数计算
    optimizer.zero_grad()
    # 进行反向传播前后的loss缩放、参数更新
    scaler.scale(loss).backward()    # loss缩放并反向传播
    scaler.step(optimizer)    # 更新参数（自动unscaling）
    scaler.update()    # 基于动态Loss Scale更新loss_scaling系数

六. 未知错误

E39999: Inner Error, Please contact support engineer!
E39999  An exception occurred during AICPU execution, stream_id:78, task_id:742, errcode:21008, msg:inner error[FUNC:ProcessAicpuErrorInfo][FILE:device_error_proc.cc][LINE:673]
        TraceBack (most recent call last):
        Kernel task happen error, retCode=0x2a, [aicpu exception].[FUNC:PreCheckTaskErr][FILE:task.cc][LINE:1068]
        Aicpu kernel execute failed, device_id=0, stream_id=78, task_id=742.[FUNC:PrintAicpuErrorInfo][FILE:task.cc][LINE:774]
        Aicpu kernel execute failed, device_id=0, stream_id=78, task_id=742, fault op_name=ScatterElements[FUNC:GetError][FILE:stream.cc][LINE:1044]
        rtStreamSynchronize execute failed, reason=[aicpu exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:49]
        op[Minimum], The Minimum op dtype is not same, type1:DT_FLOAT16, type2:DT_FLOAT[FUNC:CheckTwoInputDtypeSame][FILE:util.cc][LINE:116]
        Verifying Minimum failed.[FUNC:InferShapeAndType][FILE:infershape_pass.cc][LINE:135]
        Call InferShapeAndType for node:Minimum(Minimum) failed[FUNC:Infer][FILE:infershape_pass.cc][LINE:117]
        process pass InferShapePass on node:Minimum failed, ret:4294967295[FUNC:RunPassesOnNode][FILE:base_pass.cc][LINE:530]
        build graph failed, graph id:894, ret:1343242270[FUNC:BuildModel][FILE:ge_generator.cc][LINE:1484]
        [Build][SingleOpModel]call ge interface generator.BuildSingleOpModel failed. ge result = 1343242270[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
        [Build][Op]Fail to build op model[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        build op model failed, result = 500002[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]


DEVICE[0] PID[189368]: 
EXCEPTION TASK:
  Exception info:TGID=3114744, model id=65535, stream id=78, stream phase=SCHEDULE, task id=742, task type=aicpu kernel, recently received task id=742, recently send task id=741, task phase=RUN
  Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
    Other info[0]:time=2023-10-12-12:12:22.763.259, function=proc_aicpu_task_done, line=972, error code=0x2a 
EXCEPTION TASK:
  Exception info:TGID=3114744, model id=65535, stream id=78, stream phase=3, task id=4347, task type=aicpu kernel, recently received task id=4354, recently send task id=4346, task phase=RUN
  Message info[0]:aicpu=0,slot_id=0,report_mailbox_flag=0x5a5a5a5a,state=0x5210
    Other info[0]:time=2023-10-12-12:13:57.997.757, function=proc_aicpu_task_done, line=972, error code=0x2a
Aborted (core dumped)
(py38) [ma-user SPADE-master]$Process ForkServerProcess-2:
Traceback (most recent call last):
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 61, in wrapper
    raise exp
  File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 58, in wrapper
    func(*args, **kwargs)
  File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 268, in task_distribute
    key, func_name, detail = resource_proxy[TASK_QUEUE].get()
  File "<string>", line 2, in get
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/managers.py", line 835, in _callmethod
    kind, result = conn.recv()
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError
/home/ma-user/anaconda3/envs/py38/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 91 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

参考链接1
参考链接2：昇腾官网


http://www.niftyadmin.cn/n/5083793.html

相关文章

代码随想录算法训练营第二十天 | LeetCode 530. 二叉搜索树的最小绝对差、501. 二叉搜索树中的众数、236. 二叉树的最近公共祖先

代码随想录算法训练营第二十天 | LeetCode 530. 二叉搜索树的最小绝对差、501. 二叉搜索树中的众数、236. 二叉树的最近公共祖先

大标题 文章链接&#xff1a;二叉搜索树的最小绝对差 二叉搜索树中的众数 二叉树的最近公共祖先 视频链接&#xff1a;二叉搜索树的最小绝对差 二叉搜索树中的众数 二叉树的最近公共祖先 1. LeetCode 530. 二叉搜索树的最小绝对差 1.1 思路 因为…
阅读更多...
[C++11]花括号{}、initializer_list、auto、decltype

[C++11]花括号{}、initializer_list、auto、decltype

文章目录 1.花括号{ }的扩展2.initializer_list3.auto4.decltype5.容器的增加5.1array[useless]5.2forward_list[useless]5.3unordered_map/unordered_set5.4统一增加 6.知乎文章 1.花括号{ }的扩展 int main() {//C98花括号{ }支持 1.数组 2.结构体struct Point{int _x;int _…
阅读更多...
vue如何禁止通过页面输入路径跳转页面

vue如何禁止通过页面输入路径跳转页面

要禁止通过页面输入路径跳转页面&#xff0c;你可以使用Vue Router的导航守卫&#xff08;navigation guards&#xff09;来拦截导航并阻止不希望的跳转。 下面是一种常见的方法&#xff0c;你可以在全局导航守卫&#xff08;global navigation guards&#xff09;中实现这个功…
阅读更多...
软件测试学习（三）易用性测试、测试文档、软件安全性测试、网站测试

软件测试学习（三）易用性测试、测试文档、软件安全性测试、网站测试

目录 易用性测试 用户界面测试 优秀Ul由什么构成 符合标准和规范 直观 一致 灵活 舒适 正确 实用 为有残疾障碍的人员测试&#xff1a;辅助选项测试 测试文档 软件文档的类型 文档测试的重要性 软件安全性测试 了解黑客的动机 威胁模式分析 网站测试 网页基…
阅读更多...
回归预测、分类预测、时间序列预测 都有什么区别?

回归预测、分类预测、时间序列预测 都有什么区别?

回归预测、分类预测和时间序列预测都是统计和机器学习领域中的预测任务&#xff0c;它们在问题设置和解决的方式上有一些关键区别&#xff1a; 回归预测&#xff1a; 回归预测用于预测连续数值的输出&#xff0c;通常是实数。例如&#xff0c;预测房价、气温、销售额等连续型输…
阅读更多...
sql 注入, 盲注

sql 注入, 盲注

sql 注入, 盲注 盲注适合在页面没有任何回显时使用. 测试页面有变化, 但是没有显示任何异常错误等信息. 情景: url: http://192.168.112.200/security/read.php?id1 服务器数据库名: learn一, boolean盲注 # 盲注可能需要一个一个字符去试探, 字符串处理函数经常会用到. 比…
阅读更多...
Apache Common CLI 使用方法

Apache Common CLI 使用方法

Apache Common CLI 用于开发命令行工具&#xff0c;可以实现命令参数的解析。 一、构造所需要的参数 如果我们希望接收某个参数&#xff0c;需要按照这个方式进行构建。 方式一&#xff08;通过OptionBuilder构建&#xff09;、 Option testOption Option.builder("F…
阅读更多...
【wxWidgets 实现Window窗口系统】

【wxWidgets 实现Window窗口系统】

【wxWidgets 实现Window窗口系统】 简介效果 简介 wxWidgets 嗯 还不错,速度快,文档资源全面,轻巧,免费等等,好处一堆但是,窗口透明效果不咋地,不知道是我显卡问题还是啥设置透明全成黑色了,想折腾成JavaFx那样的界面,瞬间没感觉了,不管怎么折腾都达不到预期效果,但是可以实现…
阅读更多...
最新文章

Copyright @ 2022~2023