2017一起做网店网站,网站功能模块图,网站内的搜索怎么做的,wordpress4.9漏洞利用目录
一、引言
二、模型简介
2.1 Qwen2 模型概述
2.2 Qwen2 模型架构
三、训练与推理
3.1 Qwen2 模型训练
3.2 Qwen2 模型推理
四、总结 一、引言
刚刚写完【机器学习】Qwen1.5-14B-Chat大模型训练与推理实战 #xff0c;阿里Qwen就推出了Qwen2#x…
目录
一、引言
二、模型简介
2.1 Qwen2 模型概述
2.2 Qwen2 模型架构
三、训练与推理
3.1 Qwen2 模型训练
3.2 Qwen2 模型推理
四、总结 一、引言
刚刚写完【机器学习】Qwen1.5-14B-Chat大模型训练与推理实战 阿里Qwen就推出了Qwen2相较于Qwen1.5中0.5B、1.8B、4B、7B、14B、32B、72B、110B等8个Dense模型以及1个14BA2.7BMoE模型共计9个模型Qwen2包含了0.5B、1.5B、7B、57B-A14B和72B共计5个尺寸模型。从尺寸上来讲最关键的就是推出了57B-A14B这个更大尺寸的MoE模型有人问为什么删除了14B这个针对32G显存的常用尺寸其实对于57B-A14B剪枝一下就可以得到。
二、模型简介
2.1 Qwen2 模型概述
Qwen2对比Qwen1.5
模型尺寸将Qwen2-7B和Qwen2-72B的模型尺寸有32K提升为128K GQA分组查询注意力在Qwen1.5系列中只有32B和110B的模型使用了GQA。这一次所有尺寸的模型都使用了GQA提供GQA加速推理和降低显存占用 分组查询注意力 (Grouped Query Attention) 是一种在大型语言模型中的多查询注意力 (MQA) 和多头注意力 (MHA) 之间进行插值的方法它的目标是在保持 MQA 速度的同时实现 MHA 的质量 tie embedding针对小模型由于embedding参数量较大使用了tie embedding的方法让输入和输出层共享参数增加非embedding参数的占比 效果对比
Qwen2-72B全方位围剿Llama3-70B同时对比更大尺寸的Qwen1.5-110B也有很大提升官方表示来自于“预训练数据及训练方法的优化”。 2.2 Qwen2 模型架构
Qwen2仍然是一个典型decoder-only的transformers大模型结构主要包括文本输入层、embedding层、decoder层、输出层及损失函数
通过AutoModelForCausalLM查看Qwen1.5-7B-Chat和Qwen2-7B-Instruct的模型结构对比config.json发现 网络结构无明显变化核心网络Qwen2DecoderLayer层由32层减少为28层72B是80层Q、K、V、O隐层尺寸由4096减少为358472B是8192attention heads由32减少为2872B是64kv head由32减少为472B是8滑动窗口模型尺寸由3276832K增长为131072128K72B一样词表由151936增长为15206472B一样intermediate_sizeMLP交叉层由11008增长为1894472B是29568 可以看到其中有的参数增加有的参数减少猜想是 减少的参数并不会降低模型效果反而能增加训练和推理效率增大的参数比如MLP中的intermediate_size参数越多模型表达能力越明显。 三、训练与推理
3.1 Qwen2 模型训练
在【机器学习】Qwen1.5-14B-Chat大模型训练与推理实战 中我们采用LLaMA-Factory的webui进行训练今天我们换成命令行的方式对于LLaMA-Factory框架的部署可以参考我之前的文章
AI智能体研发之路-模型篇一大模型训练框架LLaMA-Factory在国内网络环境下的安装、部署及使用
该文在百度“LLaMA Factory 部署”词条排行第一
假设你已经基于上文部署了llama_factory的container运行进入到container中
docker exec -it llama_factory bash
在app/目录下建立run_train.sh。 CUDA_VISIBLE_DEVICES2 llamafactory-cli train \--stage sft \--do_train True \--model_name_or_path qwen/Qwen2-7B-Instruct \--finetuning_type lora \--template qwen \--flash_attn auto \--dataset_dir data \--dataset alpaca_zh \--cutoff_len 4096 \--learning_rate 5e-05 \--num_train_epochs 5.0 \--max_samples 100000 \--per_device_train_batch_size 4 \--gradient_accumulation_steps 4 \--lr_scheduler_type cosine \--max_grad_norm 1.0 \--logging_steps 10 \--save_steps 1000 \--warmup_steps 0 \--optim adamw_torch \--packing False \--report_to none \--output_dir saves/Qwen2-7B-Instruct/lora/train_2024-06-09-23-00 \--fp16 True \--lora_rank 32 \--lora_alpha 16 \--lora_dropout 0 \--lora_target q_proj,v_proj \--val_size 0.1 \--evaluation_strategy steps \--eval_steps 1000 \--per_device_eval_batch_size 2 \--load_best_model_at_end True \--plot_loss True 因为之前文章中重点讲的就是国内网络环境的LLaMA-Factory部署核心就是采用modelscope模型源代替huggingface模型源这里脚本启动后就会自动从modelscope下载指定的模型这里是qwen/Qwen2-7B-Instruct下载完后启动训练
训练数据可以通过LLaMA-Factory/data/dataset_info.json文件进行配置格式参考data目录下的其他数据文件。 比如构建成类型LLaMA-Factory/data/alpaca_zh_demo.json的格式 在LLaMA-Factory/data/dataset_info.json中复制一份进行配置 3.2 Qwen2 模型推理
Qwen2的官方文档中介绍了多种优化推理部署的方式包括基于hf transformers、vllm、llama.cpp、Ollama以及AWQ、GPTQ、GGUF等量化方式主要因为Qwen2开源的Qwen2-72B、Qwen1.5-110B远大于GLM4、Baichuan等开源的10B量级小尺寸模型。需要考虑量化、分布式推理问题。今天重点介绍Qwen2-7B-Instruct在国内网络环境下的hf transformers推理测试其他方法单开篇幅进行细致讲解。
呈上一份glm-4-9b-chat、qwen/Qwen2-7B-Instruct通用的极简代码
from modelscope import snapshot_download
from transformers import AutoTokenizer, AutoModelForCausalLM
#model_dir snapshot_download(ZhipuAI/glm-4-9b-chat)
model_dir snapshot_download(qwen/Qwen2-7B-Instruct)
#model_dir snapshot_download(baichuan-inc/Baichuan2-13B-Chat)
import torchdevice auto # 也可以通过coda:2指定GPUtokenizer AutoTokenizer.from_pretrained(model_dir,trust_remote_codeTrue)
model AutoModelForCausalLM.from_pretrained(model_dir,device_mapdevice,trust_remote_codeTrue)
print(model)prompt 介绍一下大语言模型
messages [{role: system, content: 你是一个智能助理.},{role: user, content: prompt}
]
text tokenizer.apply_chat_template(messages,tokenizeFalse,add_generation_promptTrue
)
model_inputs tokenizer([text], return_tensorspt).to(model.device)
gen_kwargs {max_length: 512, do_sample: True, top_k: 1}
with torch.no_grad():outputs model.generate(**model_inputs, **gen_kwargs)#print(tokenizer.decode(outputs[0],skip_special_tokensTrue))outputs outputs[:, model_inputs[input_ids].shape[1]:] #切除system、user等对话前缀print(tokenizer.decode(outputs[0], skip_special_tokensTrue))
generated_ids model.generate(model_inputs.input_ids,max_new_tokens512
)
generated_ids [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]response tokenizer.batch_decode(generated_ids, skip_special_tokensTrue)[0]
print(response) 该代码有几个特点 网络从modelscope下载模型文件解决通过AutoModelForCausalLM模型头下载hf模型慢的问题通用适用于glm-4-9b-chat、qwen/Qwen2-7B-Instructapply_chat_template() 注意采用 generate()替代旧方法中的chat() 。这里使用了 apply_chat_template() 函数将消息转换为模型能够理解的格式。其中的 add_generation_prompt 参数用于在输入中添加生成提示该提示指向 |im_start|assistant\n 。 tokenizer.batch_decode() 通过 tokenizer.batch_decode() 函数对响应进行解码。 运行结果 除了该极简代码我针对网络环境对官方git提供的demo代码进行了改造
cli_demo:
采用modelscope的AutoModelForCausalLM, AutoTokenizer模型头代替transformers对应的模型头进行模型自动下载
# Copyright (c) Alibaba Cloud.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.A simple command-line interactive chat demo.import argparse
import os
import platform
import shutil
from copy import deepcopy
from threading import Threadimport torch
from modelscope import AutoModelForCausalLM, AutoTokenizer
from transformers import TextIteratorStreamer
from transformers.trainer_utils import set_seedDEFAULT_CKPT_PATH Qwen/Qwen2-7B-Instruct_WELCOME_MSG \
Welcome to use Qwen2-Instruct model, type text to start chat, type :h to show command help.
(欢迎使用 Qwen2-Instruct 模型输入内容即可进行对话:h 显示命令帮助。)Note: This demo is governed by the original license of Qwen2.
We strongly advise users not to knowingly generate or allow others to knowingly generate harmful content, including hate speech, violence, pornography, deception, etc.
(注本演示受Qwen2的许可协议限制。我们强烈建议用户不应传播及不应允许他人传播以下内容包括但不限于仇恨言论、暴力、色情、欺诈相关的有害信息。)_HELP_MSG \
Commands::help / :h Show this help message 显示帮助信息:exit / :quit / :q Exit the demo 退出Demo:clear / :cl Clear screen 清屏:clear-history / :clh Clear history 清除对话历史:history / :his Show history 显示对话历史:seed Show current random seed 显示当前随机种子:seed N Set random seed to N 设置随机种子:conf Show current generation config 显示生成配置:conf keyvalue Change generation config 修改生成配置:reset-conf Reset generation config 重置生成配置_ALL_COMMAND_NAMES [help, h, exit, quit, q, clear, cl, clear-history, clh, history, his,seed, conf, reset-conf,
]def _setup_readline():try:import readlineexcept ImportError:return_matches []def _completer(text, state):nonlocal _matchesif state 0:_matches [cmd_name for cmd_name in _ALL_COMMAND_NAMES if cmd_name.startswith(text)]if 0 state len(_matches):return _matches[state]return Nonereadline.set_completer(_completer)readline.parse_and_bind(tab: complete)def _load_model_tokenizer(args):tokenizer AutoTokenizer.from_pretrained(args.checkpoint_path, resume_downloadTrue,)if args.cpu_only:device_map cpuelse:device_map automodel AutoModelForCausalLM.from_pretrained(args.checkpoint_path,torch_dtypeauto,device_mapdevice_map,resume_downloadTrue,).eval()model.generation_config.max_new_tokens 2048 # For chat.return model, tokenizerdef _gc():import gcgc.collect()if torch.cuda.is_available():torch.cuda.empty_cache()def _clear_screen():if platform.system() Windows:os.system(cls)else:os.system(clear)def _print_history(history):terminal_width shutil.get_terminal_size()[0]print(fHistory ({len(history)}).center(terminal_width, ))for index, (query, response) in enumerate(history):print(fUser[{index}]: {query})print(fQWen[{index}]: {response})print( * terminal_width)def _get_input() - str:while True:try:message input(User ).strip()except UnicodeDecodeError:print([ERROR] Encoding error in input)continueexcept KeyboardInterrupt:exit(1)if message:return messageprint([ERROR] Query is empty)def _chat_stream(model, tokenizer, query, history):conversation [{role: system, content: You are a helpful assistant.},]for query_h, response_h in history:conversation.append({role: user, content: query_h})conversation.append({role: assistant, content: response_h})conversation.append({role: user, content: query})inputs tokenizer.apply_chat_template(conversation,add_generation_promptTrue,return_tensorspt,)inputs inputs.to(model.device)streamer TextIteratorStreamer(tokenizertokenizer, skip_promptTrue, timeout60.0, skip_special_tokensTrue)generation_kwargs dict(input_idsinputs,streamerstreamer,)thread Thread(targetmodel.generate, kwargsgeneration_kwargs)thread.start()for new_text in streamer:yield new_textdef main():parser argparse.ArgumentParser(descriptionQWen2-Instruct command-line interactive chat demo.)parser.add_argument(-c, --checkpoint-path, typestr, defaultDEFAULT_CKPT_PATH,helpCheckpoint name or path, default to %(default)r)parser.add_argument(-s, --seed, typeint, default1234, helpRandom seed)parser.add_argument(--cpu-only, actionstore_true, helpRun demo with CPU only)args parser.parse_args()history, response [], model, tokenizer _load_model_tokenizer(args)orig_gen_config deepcopy(model.generation_config)_setup_readline()_clear_screen()print(_WELCOME_MSG)seed args.seedwhile True:query _get_input()# Process commands.if query.startswith(:):command_words query[1:].strip().split()if not command_words:command else:command command_words[0]if command in [exit, quit, q]:breakelif command in [clear, cl]:_clear_screen()print(_WELCOME_MSG)_gc()continueelif command in [clear-history, clh]:print(f[INFO] All {len(history)} history cleared)history.clear()_gc()continueelif command in [help, h]:print(_HELP_MSG)continueelif command in [history, his]:_print_history(history)continueelif command in [seed]:if len(command_words) 1:print(f[INFO] Current random seed: {seed})continueelse:new_seed_s command_words[1]try:new_seed int(new_seed_s)except ValueError:print(f[WARNING] Fail to change random seed: {new_seed_s!r} is not a valid number)else:print(f[INFO] Random seed changed to {new_seed})seed new_seedcontinueelif command in [conf]:if len(command_words) 1:print(model.generation_config)else:for key_value_pairs_str in command_words[1:]:eq_idx key_value_pairs_str.find()if eq_idx -1:print([WARNING] format: keyvalue)continueconf_key, conf_value_str key_value_pairs_str[:eq_idx], key_value_pairs_str[eq_idx 1:]try:conf_value eval(conf_value_str)except Exception as e:print(e)continueelse:print(f[INFO] Change config: model.generation_config.{conf_key} {conf_value})setattr(model.generation_config, conf_key, conf_value)continueelif command in [reset-conf]:print([INFO] Reset generation config)model.generation_config deepcopy(orig_gen_config)print(model.generation_config)continueelse:# As normal query.pass# Run chat.set_seed(seed)_clear_screen()print(f\nUser: {query})print(f\nQwen2-Instruct: , end)try:partial_text for new_text in _chat_stream(model, tokenizer, query, history):print(new_text, end, flushTrue)partial_text new_textresponse partial_textprint()except KeyboardInterrupt:print([WARNING] Generation interrupted)continuehistory.append((query, response))if __name__ __main__:main()web_demo.py:
同上采用modelscope代替transformers饮用AutoModelForCausalLM, AutoTokenizer解决模型下载问题
输入参数加入-g指定运行的GPU
# Copyright (c) Alibaba Cloud.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.A simple web interactive chat demo based on gradio.from argparse import ArgumentParser
from threading import Threadimport gradio as gr
import torch
from modelscope import AutoModelForCausalLM, AutoTokenizer
from transformers import TextIteratorStreamerDEFAULT_CKPT_PATH Qwen/Qwen2-7B-Instructdef _get_args():parser ArgumentParser()parser.add_argument(-c, --checkpoint-path, typestr, defaultDEFAULT_CKPT_PATH,helpCheckpoint name or path, default to %(default)r)parser.add_argument(--cpu-only, actionstore_true, helpRun demo with CPU only)parser.add_argument(--share, actionstore_true, defaultFalse,helpCreate a publicly shareable link for the interface.)parser.add_argument(--inbrowser, actionstore_true, defaultFalse,helpAutomatically launch the interface in a new tab on the default browser.)parser.add_argument(--server-port, typeint, default18003,helpDemo server port.)parser.add_argument(--server-name, typestr, default127.0.0.1,helpDemo server name.)parser.add_argument(-g,--gpus,typestr,defaultauto,helpset gpu numbers)args parser.parse_args()return argsdef _load_model_tokenizer(args):tokenizer AutoTokenizer.from_pretrained(args.checkpoint_path, resume_downloadTrue,)if args.cpu_only:device_map cpuelif args.gpusauto:device_map args.gpuselse:device_map cuda:args.gpusmodel AutoModelForCausalLM.from_pretrained(args.checkpoint_path,torch_dtypeauto,device_mapdevice_map,resume_downloadTrue,).eval()model.generation_config.max_new_tokens 2048 # For chat.return model, tokenizerdef _chat_stream(model, tokenizer, query, history):conversation [{role: system, content: You are a helpful assistant.},]for query_h, response_h in history:conversation.append({role: user, content: query_h})conversation.append({role: assistant, content: response_h})conversation.append({role: user, content: query})inputs tokenizer.apply_chat_template(conversation,add_generation_promptTrue,return_tensorspt,)inputs inputs.to(model.device)streamer TextIteratorStreamer(tokenizertokenizer, skip_promptTrue, timeout60.0, skip_special_tokensTrue)generation_kwargs dict(input_idsinputs,streamerstreamer,)thread Thread(targetmodel.generate, kwargsgeneration_kwargs)thread.start()for new_text in streamer:yield new_textdef _gc():import gcgc.collect()if torch.cuda.is_available():torch.cuda.empty_cache()def _launch_demo(args, model, tokenizer):def predict(_query, _chatbot, _task_history):print(fUser: {_query})_chatbot.append((_query, ))full_response response for new_text in _chat_stream(model, tokenizer, _query, history_task_history):response new_text_chatbot[-1] (_query, response)yield _chatbotfull_response responseprint(fHistory: {_task_history})_task_history.append((_query, full_response))print(fQwen2-Instruct: {full_response})def regenerate(_chatbot, _task_history):if not _task_history:yield _chatbotreturnitem _task_history.pop(-1)_chatbot.pop(-1)yield from predict(item[0], _chatbot, _task_history)def reset_user_input():return gr.update(value)def reset_state(_chatbot, _task_history):_task_history.clear()_chatbot.clear()_gc()return _chatbotwith gr.Blocks() as demo:gr.Markdown(\
p aligncenterimg srchttps://qianwen-res.oss-accelerate-overseas.aliyuncs.com/logo_qwen2.png styleheight: 80px/p)gr.Markdown(centerfont size8Qwen2 Chat Bot/center)gr.Markdown(\
centerfont size3This WebUI is based on Qwen2-Instruct, developed by Alibaba Cloud. \
(本WebUI基于Qwen2-Instruct打造实现聊天机器人功能。)/center)gr.Markdown(\
centerfont size4
Qwen2-7B-Instruct a hrefhttps://modelscope.cn/models/qwen/Qwen2-7B-Instruct/summary /a |
a hrefhttps://huggingface.co/Qwen/Qwen2-7B-Instruct/anbsp
Qwen2-72B-Instruct a hrefhttps://modelscope.cn/models/qwen/Qwen2-72B-Instruct/summary /a |
a hrefhttps://huggingface.co/Qwen/Qwen2-72B-Instruct/anbsp
nbspa hrefhttps://github.com/QwenLM/Qwen2Github/a/center)chatbot gr.Chatbot(labelQwen2-Instruct, elem_classescontrol-height)query gr.Textbox(lines2, labelInput)task_history gr.State([])with gr.Row():empty_btn gr.Button( Clear History (清除历史))submit_btn gr.Button( Submit (发送))regen_btn gr.Button(️ Regenerate (重试))submit_btn.click(predict, [query, chatbot, task_history], [chatbot], show_progressTrue)submit_btn.click(reset_user_input, [], [query])empty_btn.click(reset_state, [chatbot, task_history], outputs[chatbot], show_progressTrue)regen_btn.click(regenerate, [chatbot, task_history], [chatbot], show_progressTrue)gr.Markdown(\
font size2Note: This demo is governed by the original license of Qwen2. \
We strongly advise users not to knowingly generate or allow others to knowingly generate harmful content, \
including hate speech, violence, pornography, deception, etc. \
(注本演示受Qwen2的许可协议限制。我们强烈建议用户不应传播及不应允许他人传播以下内容\
包括但不限于仇恨言论、暴力、色情、欺诈相关的有害信息。))demo.queue().launch(shareargs.share,inbrowserargs.inbrowser,server_portargs.server_port,server_nameargs.server_name,)def main():args _get_args()model, tokenizer _load_model_tokenizer(args)_launch_demo(args, model, tokenizer)if __name__ __main__:main()四、总结
本文首先对Qwen2模型概述以及模型架构进行讲解接着基于llama_factory命令行的方式进行模型训练演示最后基于hf transformers进行模型推理的讲解。过程中排了好几个坑呈上的代码保证在国内网络环境下是可运行的。希望能帮助到大家。喜欢的话关注三连噢。 如果您还有时间可以看看我的其他文章
《AI—工程篇》
AI智能体研发之路-工程篇一Docker助力AI智能体开发提效
AI智能体研发之路-工程篇二Dify智能体开发平台一键部署
AI智能体研发之路-工程篇三大模型推理服务框架Ollama一键部署
AI智能体研发之路-工程篇四大模型推理服务框架Xinference一键部署
AI智能体研发之路-工程篇五大模型推理服务框架LocalAI一键部署
《AI—模型篇》
AI智能体研发之路-模型篇一大模型训练框架LLaMA-Factory在国内网络环境下的安装、部署及使用
AI智能体研发之路-模型篇二DeepSeek-V2-Chat 训练与推理实战
AI智能体研发之路-模型篇三中文大模型开、闭源之争
AI智能体研发之路-模型篇四一文入门pytorch开发
AI智能体研发之路-模型篇五pytorch vs tensorflow框架DNN网络结构源码级对比
AI智能体研发之路-模型篇六【机器学习】基于tensorflow实现你的第一个DNN网络
AI智能体研发之路-模型篇七【机器学习】基于YOLOv10实现你的第一个视觉AI大模型
AI智能体研发之路-模型篇八【机器学习】Qwen1.5-14B-Chat大模型训练与推理实战
AI智能体研发之路-模型篇九【机器学习】GLM4-9B-Chat大模型/GLM-4V-9B多模态大模型概述、原理及推理实战
《AI—Transformers应用》
【AI大模型】Transformers大模型库一Tokenizer
【AI大模型】Transformers大模型库二AutoModelForCausalLM
【AI大模型】Transformers大模型库三特殊标记special tokens
【AI大模型】Transformers大模型库四AutoTokenizer
