搜索到
66
篇与
学习笔记
的结果
-
docker容器中执行脚本被killed，修改wsl2内存限制大小 在容器中执行脚本，被killed。大概猜想是内存不够，本机是16G、docker限制8G。查看日志#查看当前 Docker 使用的 WSL 发行版 wsl -l -v #进入 docker 的 WSL 宿主机（一般是 docker-desktop） wsl -d docker-desktop #查看 dmesg 日志（内核日志） dmesg | grep -i kill # 或者 dmesg -T | grep -i "killed process"果真，那就修改一下内存限制再试试吧。修改wsl2限制内存定位到你的 用户目录（比如 C:Users你的用户名）（可以win + r 输入 %UserProfile% ）,打开用户目录，创建.wslconfig文件。C:Users<你的用户名>.wslconfig添加或修改以下内容：[wsl2] memory=12GB swap=4G processors=6memory=12GB # 分配给 WSL2 的最大内存processors=6 # 可选，限制使用的 CPU 核心数swap=8GB # 可选，设置 swap 大小保存后，重启 WSLwsl --shutdown重启 Docker Desktop。修改之前修改之后虽然还是没什么卵用，去线上跑吧。垃圾 -
-
Python-openAI 助手接口封装(含创建助手、文件上传、线程、消息、运行) 最近的项目中用到GPT，简单封装一下。包含创建助手、上传文件、创建线程、创建消息线程、运行线程、删除等操作。我是使用的异步，同步大差不大，去掉async、await就行了，调用方法都是一样的。官方API接口：https://platform.openai.com/docs/api-reference/assistants核心代码隐藏内容，请前往内页查看详情测试运行api_key = "sk----------" assistant_id = "asst_R3KqDZNg4QPzxd6U56INh0Cd" thread_id = "thread_vWZ9QrPBPTDHrtxIiSIof3hs" file_assistant_gpt = FileAssistant(api_key,assistant_id, thread_id) await file_assistant_gpt.run_file_analysis("data.txt","请帮我分析该文件...")
-
python 异步网络请求封装函数 带重试机制，支持get、post python基于httpx的异步网络请求，带重试机制，支持post、get，自定义header、携带cookie、等功能。函数import asyncio from typing import Optional import httpx ''' 带重试机制的网络请求函数 post、get ''' # get请求 重试机制 async def curl_get( client: httpx.AsyncClient, url: str, id:int = 0, cookie: Optional[dict] = None, headers: Optional[dict] = None, retries: int = 3): # 设置重试次数 for attempt in range(1,retries + 1): try: response = await client.get(url, cookies=cookie, headers=headers) response.raise_for_status() # 如果请求失败，抛出异常 return {"code": 1,"id":id, "response":response,"attempt":attempt} # 返回解析后的 JSON 数据 except httpx.HTTPStatusError as e: if attempt < retries: await asyncio.sleep(1) # 重试前暂停1秒 else: return {"code": 0,"id":id, "error": f"请求失败：{e.response.status_code}","attempt":attempt} except httpx.RequestError as e: print("网络错误",e) # 捕获网络错误（例如请求超时、连接错误等） if attempt < retries: await asyncio.sleep(2) # 重试前暂停1秒 else: return {"code": 0, "id":id,"error": f"网络错误 [已重试{str(attempt)}次]：{str(e)}","attempt":attempt} except Exception as e: if attempt < retries: await asyncio.sleep(1) # 重试前暂停1秒 else: return {"code": 0, "id": id, "error": f"请求错误 [已重试{str(attempt)}次]：{str(e)}", "attempt": attempt} return {"code": 0,"id":id, "error": "重试次数已达上限","attempt":attempt} # post请求 重试机制 async def curl_post( client: httpx.AsyncClient, url: str, data = None, cookie: Optional[dict] = None, headers: Optional[dict] = None, retries: int = 3): # 设置重试次数 for attempt in range(1,retries + 1): try: response = await client.post(url=url, cookies=cookie, headers=headers, data=data) response.raise_for_status() # 如果请求失败，抛出异常 html = response.text return {"code": 1, 'response':response, "data":html , "attempt": attempt } # 返回解析后的 JSON 数据 except httpx.HTTPStatusError as e: # 捕获 HTTP 错误（例如 404 或 500 错误） if attempt < retries: await asyncio.sleep(1) # 重试前暂停1秒 else: return {"code": 0, "error": f"请求失败：{e.response.status_code}", "attempt": attempt} except httpx.RequestError as e: # 捕获网络错误（例如请求超时、连接错误等） if attempt < retries: await asyncio.sleep(1) # 重试前暂停1秒 else: return {"code": 0,"error": f"请求错误 [{str(attempt)}] {str(e)}", "attempt": attempt} except Exception as e: if attempt < retries: await asyncio.sleep(1) # 重试前暂停1秒 else: return {"code": 0, "id": id, "error": f"请求错误 [{str(attempt)}]：{str(e)}", "attempt": attempt} return {"code": 0, "error": "重试次数已达上限","attempt":attempt}使用async with httpx.AsyncClient() as client: reslut = await curl_get(client, url) if reslut.get("code") != 1: return reslut.get("error") response = reslut.get("response") json = response.json()
-
docker-R语言多版本安装，在python中使用R容器执行脚本 .env# 新增R版本 4.2.1 R_NAME_421=r_4_2_1 # 容器/镜像 名称 R_VERSION_421=4.2.1 # 版本号 R_PORT_421=8780 # 新增R版本 4.3.0 R_NAME_430=r_4_3_0 R_VERSION_430=4.3.0 R_PORT_430=8781 # 新增R版本 4.4.2 R_NAME_442=r_4_4_2 R_VERSION_442=4.4.2 R_PORT_442=8782docker-compose.yml # R 服务 4.2.1 r_4_2_1: build: context: . dockerfile: ${R_DOCKERFILE} # dockerfile公共文件 args: FROM_IMAGE: rocker/r-ver:${R_VERSION_421} # 指定镜像版本 R_BIOC_MIRROR: ${R_BIOC_MIRROR} # 公共 镜像源 R_CRAN: ${R_CRAN} container_name: ${R_NAME_421} # 指定版本名称 image: ${R_NAME_421}-image # 指定镜像名称 ports: - "${R_PORT_421}:${R_PORT_421}" # 指定端口 volumes: - ./R:/workspace/R - ./static/matrix:/workspace/static/matrix - .env:/workspace/.env # 挂载 .env 文件 #- ./store/docker/r-library/${R_NAME_442}:/usr/local/lib/R/site-library # 持久化 R 包的安装位置 environment: - R_HOME=/usr/local/lib/R command: Rscript R/http/start.R # 启动API服务 restart: unless-stopped networks: - backend # R 服务 4.3.0 r_4_3_0: build: context: . dockerfile: ${R_DOCKERFILE} # dockerfile公共文件 args: FROM_IMAGE: rocker/r-ver:${R_VERSION_430} # 指定镜像版本 R_BIOC_MIRROR: ${R_BIOC_MIRROR} # 公共 镜像源 R_CRAN: ${R_CRAN} container_name: ${R_NAME_430} # 指定版本名称 image: ${R_NAME_430}-image # 指定镜像名称 ports: - "${R_PORT_430}:${R_PORT_430}" # 指定端口 volumes: - ./R:/workspace/R - ./static/matrix:/workspace/static/matrix - .env:/workspace/.env # 挂载 .env 文件 #- ./store/docker/r-library/${R_NAME_430}:/usr/local/lib/R/site-library # 持久化 R 包的安装位置 environment: - R_HOME=/usr/local/lib/R command: Rscript R/http/start.R # 启动API服务 restart: unless-stopped networks: - backend # R 服务 4.4.2 r_4_4_2: build: context: . dockerfile: ${R_DOCKERFILE} # dockerfile公共文件 args: FROM_IMAGE: rocker/r-ver:${R_VERSION_442} # 指定镜像版本 R_BIOC_MIRROR: ${R_BIOC_MIRROR} # 公共 镜像源 R_CRAN: ${R_CRAN} container_name: ${R_NAME_442} # 指定版本名称 image: ${R_NAME_442}-image # 指定镜像名称 ports: - "${R_PORT_442}:${R_PORT_442}" # 指定端口 volumes: - ./R:/workspace/R - ./static/matrix:/workspace/static/matrix - .env:/workspace/.env # 挂载 .env 文件 #- ./store/docker/r-library/${R_VERSION_442}:/usr/local/lib/R/site-library # 持久化 R 包的安装位置 environment: - R_HOME=/usr/local/lib/R command: Rscript R/http/start.R # 启动API服务 restart: unless-stopped networks: - backend dockerfileARG FROM_IMAGE=rocker/r-ver:4.4.1 FROM ${FROM_IMAGE} ARG R_BIOC_MIRROR=https://mirrors.tuna.tsinghua.edu.cn/bioconductor ARG R_CRAN=https://mirrors.tuna.tsinghua.edu.cn/CRAN/ # 设置镜像源为清华大学的 CRAN 镜像和 Bioconductor 镜像 RUN echo "options(BioC_mirror = '${R_BIOC_MIRROR}')" >> /usr/local/lib/R/etc/Rprofile.site RUN echo "options(repos = c(CRAN = '${R_CRAN}'))" >> /usr/local/lib/R/etc/Rprofile.site ## 安装系统依赖包 RUN sed -i 's|http://archive.ubuntu.com/ubuntu/|https://mirrors.tuna.tsinghua.edu.cn/ubuntu/|' /etc/apt/sources.list && \ apt-get update && \ apt-get install -y \ libcurl4-openssl-dev \ libssl-dev \ libxml2-dev \ libfontconfig1-dev \ libharfbuzz-dev \ libfribidi-dev \ libpango1.0-dev \ libx11-dev \ libxt-dev \ libjpeg-dev \ libpng-dev \ libtiff-dev \ libbz2-dev \ libsodium-dev \ && rm -rf /var/lib/apt/lists/* \ # 安装中文字体 RUN apt-get update && apt-get install -y fonts-wqy-zenhei fonts-wqy-microhei ttf-mscorefonts-installer fontconfig # 查看系统中的中文字体：fc-list :lang=zh # 报错：dpkg 进程被中断了； 修复：dpkg --configure -a ## 设定工作目录 WORKDIR /workspace ## 常用包安装脚本 COPY R/docker/install_packages.R /workspace/install_packages.R # 验证文件是否被成功复制 RUN ls -l /workspace/install_packages.R ## 执行(耗时) RUN Rscript /workspace/install_packages.R ## 设定工作目录 WORKDIR /workspace # ## 保持容器在后台运行 CMD ["tail", "-f", "/dev/null"] install_packages.R # 安装常用的包 install.packages('pheatmap') install.packages('BiocManager') options(timeout = 600) install_missing_packages <- function(packages) { if (!requireNamespace("BiocManager", quietly = TRUE)) { install.packages("BiocManager") } for (package in packages) { if (!requireNamespace(package, quietly = TRUE)) { cat(paste("开始安装", package, "...\n")) BiocManager::install(package, ask = FALSE) } else { cat(paste( packageVersion(package), package, "已安装.\n" )) } } } # 安装常用的包 packages <- c('GEOquery', 'ggplot2', 'readxl', 'readr', 'openxlsx', 'dplyr', 'clusterProfiler', 'limma', 'DESeq2', 'ggtree', 'org.Hs.eg.db','org.Mm.eg.db','org.Rn.eg.db','GOplot','stringr', 'enrichplot', 'treeio', 'WGCNA', 'igraph','plumber','jsonlite','dotenv','remotes') # 安装缺失的 R 包 install_missing_packages(packages) # 针对某个版本中的某些包进行检测安装 r_version <- getRversion() cat(paste("检测R版本", r_version , "...\n")) if(r_version == "4.2.1"){ if (!requireNamespace("ggtree", quietly = TRUE)) { remotes::install_github("YuLab-SMU/ggtree") } if (!requireNamespace("clusterProfiler", quietly = TRUE)) { remotes::install_github("ctlab/fgsea") BiocManager::install("clusterProfiler") } } if(r_version == "4.3.0"){ if (!requireNamespace("ggtree", quietly = TRUE) && !requireNamespace("limma", quietly = TRUE) && !requireNamespace("org.Hs.eg.db", quietly = TRUE) ) { options(BioC_mirror = "https://bioconductor.org") install_missing_packages(packages) } if (!requireNamespace("ggtree", quietly = TRUE)) { remotes::install_github("YuLab-SMU/ggtree") } if (!requireNamespace("clusterProfiler", quietly = TRUE)) { remotes::install_github("ctlab/fgsea") BiocManager::install("clusterProfiler") } } 如何在Python中使用R容器，执行R脚本？搭建好后可以通过接口的方式运行R脚本也可以使用py-docker操作容器（线上可能需要挂载docker守护进程）接口服务http.R if (!requireNamespace("plumber", quietly = TRUE)) { cat(paste("开始安装", "plumber", "...\n")) BiocManager::install("plumber", ask = FALSE) } if (!requireNamespace("jsonlite", quietly = TRUE)) { cat(paste("开始安装", "jsonlite", "...\n")) BiocManager::install("jsonlite", ask = FALSE) } if (!requireNamespace("dotenv", quietly = TRUE)) { cat(paste("开始安装", "dotenv", "...\n")) BiocManager::install("dotenv", ask = FALSE) } cat("当前工作目录",getwd(),"\n") # 加载 dotenv 包 library(dotenv) library(plumber) # 读取 .env 文件 load_dot_env(".env") #print(Sys.getenv()) # 获取 R 版本号 r_version <- getRversion() cat("R版本:", as.character(r_version), "\n") # 将 R 版本号转换为 .env 文件中的键名格式 例如：4.2.1 -> 421 r_version_key <- gsub("\\.", "", as.character(r_version)) # 构建端口号的键名 port_key <- paste0("R_PORT_", r_version_key) # 从 .env 文件中获取端口号 port <- as.integer(Sys.getenv(port_key)) if (is.na(port)) { # 如果端口号不存在，使用默认端口 port <- 8006 # 默认端口 warning("未找到当前R版本的端口配置(",port_key,") 使用默认端口：", port) }else{ cat("找到当前 R 版本的端口配置：", port, "\n") } # 创建 plumber API pr <- plumb("R/http/app.R") # 启动 plumber 服务 pr$run(host = "0.0.0.0", port = port) app.Rlibrary(jsonlite) # 定义一个 API 端点，执行 R 脚本 #* 执行 R 脚本 #* @post /run_r_script function(req, script = "", param = "") { # 调用执行脚本的函数 result <- run_script(script, param) return(result) } #* 执行系统命令 post #* @post /run_command function(req, command = "") { if (command == "") { return(list(code = unbox(0),msg = unbox("未提供命令") )) } # 执行系统命令 result <- tryCatch({ output <- system(command, intern = TRUE) list(code = unbox(1), msg = unbox("命令执行成功"), data = output) }, error = function(e) { list(code = unbox(0), msg = unbox(e$message) ) }) return(result) } # 执行 R 脚本的函数 run_script <- function(script_path, params) { setwd("/workspace") # 检查脚本文件是否存在 if (!file.exists(script_path)) { return( list( code = unbox(0), msg = unbox("脚本文件不存在"), data = getwd() )) } # 将参数转换为 R 对象 #params <- jsonlite::fromJSON(params) # 构造命令行 command <- paste("Rscript", script_path, shQuote(params)) # 在独立环境中执行脚本 result <- tryCatch({ #run_result <- source(script_path) # 执行结果 run_result <- system(command, intern = TRUE) print(run_result) # 获取退出状态 exit_status <- attr(run_result, "status") exit_status <- ifelse(is.null(exit_status), 0, exit_status) print(cat("退出状态：",exit_status,"\n")) if(exit_status == 1){ # 将 run_result 转换为字符串 result_string <- paste(run_result, collapse = "\n") #print(result_string) list( code = unbox(0), # 使用 unbox() 将 code 转换为标量值 msg = unbox(result_string), data = run_result, # 捕获的输出 output = params ) }else{ #run_result <- capture.output(source(script_path)) list( code = unbox(1), # 使用 unbox() 将 code 转换为标量值 msg = unbox("脚本执行成功"), data = run_result, # 捕获的输出 output = params ) } }, error = function(e) { print(e) list( code = unbox(0), msg = unbox("脚本执行失败"), data = NULL, output = unbox(e$message) ) }) return(result) }
-
Docker容器加入已有网络 version: '3.8' services: # FastAPI 服务 fastapi: build: context: . dockerfile: Dockerfile container_name: ${FASTAPI_CONTAINER_NAME} image: ${FASTAPI_IMAGE} # 启动web服务和队列 command: /bin/bash -c "uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4 & arq main_queue.WorkerSettings" ports: - "${FASTAPI_PORT}:8000" # depends_on: # - redis volumes: - .:/app networks: - backend # 加入一个外部网络，比如 dnmp_default 下有redis、mysql，可以直接通过 容器名称:内部端口 访问。 - dnmp_default networks: backend: driver: bridge dnmp_default: # 声明外部网络 external: true
-
-
两句命令查看电脑上已连接的wifi以及密码 附python代码 打开终端，输入两条命令，即可查看当前电脑上所有已保存的wifi以及密码可以列出已保存的wifi网络名netsh wlan show profile打印出WiFi信息（包含密码）netsh wlan show profile name=Wifi网络名字 key=clear使用python代码获取 import subprocess cmd1 = "netsh wlan show profile" cmd2 = "netsh wlan show profile name={wifi} key=clear" def get_wifi(): # 执行 查看wifi列表 命令 reslut = subprocess.run(cmd1.split() , capture_output= True,text=True) if reslut.returncode == 0: output_lines = reslut.stdout.split("\n") # 获取到所有wifi的名称 wifi_profiles = [ line.split(":")[1].strip() for line in output_lines if "所有用户配置文件" in line] for wifi_profile in wifi_profiles: # 查询每个wifi的密码 cmd = cmd2.replace("{wifi}", wifi_profile) wifikey = subprocess.run(cmd.split() , capture_output= True,text=True) if wifikey.returncode == 0: pwds = [line.split(":")[1].strip() for line in wifikey.stdout.split("\n") if "关键内容" in line] for pwd in pwds: print(f'WIFI: {wifi_profile},密码{pwd}') else: print(f'WIFI: {wifi_profile} 获取失败') else: print("获取WIFI配置失败") if __name__ == '__main__': get_wifi()
-
抖音 记一次 X-bogus 逆向分析 记录一次抖音 X-Bogus 研究过程隐藏内容，请前往内页查看详情{dotted startColor="#ff6c6c" endColor="#1989fa"/}{dotted startColor="#ff6c6c" endColor="#1989fa"/}将X-Bogus放入实际案例中测试导入js库import execjs with open("1.js") as f: js_code = f.read() f.close() js_compile = execjs.compile(js_code)接口 -> 获取粉丝列表https://www.douyin.com/aweme/v1/web/user/follower/list/Cookiecookies = { "stream_player_status_params": "", "sid_guard": "", }参数params = { "device_platform": "webapp", "aid": "6383", "channel": "channel_pc_web", "user_id": "XXXX", "sec_user_id": "XXX", "offset": "0", "min_time": "0", "max_time": "1710210938", "count": "20", "source_type": "1", "gps_access": "0", "address_book_access": "0", "pc_client_type": "1", "version_code": "170400", "version_name": "17.4.0", "cookie_enabled": "true", "screen_width": "1440", "screen_height": "900", "browser_language": "zh-CN", "browser_platform": "MacIntel", "browser_name": "Chrome", "browser_version": "122.0.0.0", "browser_online": "true", "engine_name": "Blink", "engine_version": "122.0.0.0", "os_name": "Mac OS", "os_version": "10.15.7", "cpu_core_num": "4", "device_memory": "8", "platform": "PC", "downlink": "10", "effective_type": "4g", "round_trip_time": "100", "webid": "XXX", "msToken": "1", }请求结果 成功总结这里的粉丝分页不是 offset ，而是 max_time 。这个值可以取返回参数中的 min_timecookie只用到了 stream_player_status_params 和 sid_guardmsToken不校验，但是参与生成XB。其他参数没有测试{dotted startColor="#ff6c6c" endColor="#1989fa"/}小白研究，欢迎大佬指教小白研究，欢迎大佬指教小白研究，欢迎大佬指教
-
docker-dnmp环境中安装hyperf框架 安装PHP8.2以及相关扩展.env设置php根据框架要求将php版本设置为 8.2 ，以及安装所需的扩展 opcache,redis,swoole,pcntl PHP80_VERSION=8.2.12 PHP80_PHP_CONF_FILE=./services/php80/php.ini PHP80_FPM_CONF_FILE=./services/php80/php-fpm.conf PHP80_LOG_DIR=./logs/php80 PHP80_EXTENSIONS=pdo_mysql,mysqli,mbstring,gd,curl,opcache,redis,swoole,pcntldocker-compose.yml 端口映射与暴露我设置的是9800端口 php80: build: context: ./services/php80 args: PHP_VERSION: php:${PHP80_VERSION}-fpm-alpine CONTAINER_PACKAGE_URL: ${CONTAINER_PACKAGE_URL} COMPOSER_URL: ${COMPOSER_URL} PHP_EXTENSIONS: ${PHP80_EXTENSIONS} TZ: "$TZ" container_name: php80 expose: - 9501 - 9800 ports: - "9800:9800" volumes: - ${SOURCE_DIR}:/www/:rw - ${PHP80_PHP_CONF_FILE}:/usr/local/etc/php/php.ini:ro - ${PHP80_FPM_CONF_FILE}:/usr/local/etc/php-fpm.d/www.conf:rw - ${PHP80_LOG_DIR}:/var/log/php - ${DATA_DIR}/composer:/tmp/composer restart: always cap_add: - SYS_PTRACE networks: - default设置完成后运行进入到php8的控制台docker exec -it php80 /bin/sh进入www目录，安装框架composer create-project hyperf/hyperf-skeleton 将端口改成9800hyperf-skeleton/config/autoload/server.php 'mode' => SWOOLE_PROCESS, 'servers' => [ [ 'name' => 'http', 'type' => Server::SERVER_HTTP, 'host' => '0.0.0.0', 'port' => 9800, # 这里 'sock_type' => SWOOLE_SOCK_TCP, 'callbacks' => [ Event::ON_REQUEST => [Hyperf\HttpServer\Server::class, 'onRequest'], ], 'options' => [ // Whether to enable request lifecycle event 'enable_request_lifecycle' => false, ], ], ],运行php bin/hyperf.php start访问 0.0.0.0:9800
