Skip to content

weibo表无法写入,user表正常 #693

@SemiThin

Description

@SemiThin
  • 问:请您指明哪个版本出了bug(github版/PyPi版/全部)?

答:github

  • 问:您使用的是否是最新的程序(是/否)?

答:是

  • 问:爬取任意用户都会复现此bug吗(是/否)?

答:是

  • 问:若只有爬特定微博时才出bug,能否提供出错微博的weibo_id或url(非必填)?

答:1987241375 蘑菇小牙牙 2026-02-24 14:15 或者 5371906414 佩奇线报 2026-02-24 14:15

  • 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?

答:

{
"user_id_list": "user_id_list.txt",
"filter": 0,
"since_date": "2026-02-24",
"end_date": "now",
"random_wait_pages": [
1,
5
],
"random_wait_seconds": [
6,
10
],
"global_wait": [
[
1000,
3600
],
[
500,
2000
]
],
"write_mode": [
"mysql"
],
"pic_download": 0,
"video_download": 0,
"file_download_timeout": [
5,
5,
10
],
"result_dir_name": 0,
"cookie": "my cookie...",
"mysql_config": {
"host": "127.0.0.1",
"port": 3306,
"user": "root",
"password": "root",
"charset": "utf8mb4"
},
"kafka_config": {
"bootstrap-server": "127.0.0.1:9092",
"weibo_topics": [
"spider_weibo"
],
"user_topics": [
"spider_weibo"
]
},
"sqlite_config": "weibo.db",
"mongo_config": {
"connection_string": "mongodb://admin:password@localhost:27017/weibo",
"dba_name": "",
"dba_password": ""
},
"post_config": {
"api_url": "",
"api_token": ""
}
}

  • 问:如果方便,请您描述bug详情,如果代码报错,最好附上错误提示。

答:

ubuntu @ ubuntu in ~/www/weiboSpider on git:master x [14:14:03]
$ python3 -m weibo_spider
用户昵称: 蘑菇小牙牙
用户id: 1987241375
微博数: 539367
关注数: 2292
粉丝数: 970000


蘑菇小牙牙信息写入MySQL数据库完毕


Progress: 0%| | 0/53904 [00:00<?, ?it/s]/home/ubuntu/www/weiboSpider/weibo_spider/parser/page_parser.py:45: FutureWarning: Truth-testing of elements was a source of confusion and will always return True in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
if self.selector:
🐶东 反馈小鹿蓝蓝高钙高铁高锌DHA苏打脆片80g弹4立减,凑后【0.95】小鹿蓝蓝高钙高铁高锌DHA苏打脆片宝宝零食儿童苏打脆片/海苔味/80g凑单拍3件,同店介意慎网页链接  [组图共2张] 原图 
微博发布位置:无
发布时间:2026-02-24 13:19
发布工具:蘑菇小牙牙的小店
点赞数:20
转发数:0
评论数:34
url:https://weibo.cn/comment/QtiLt3Yye


------------------------------已获取蘑菇小牙牙(1987241375)的第1页微博------------------------------
dict can not be used as parameter
Traceback (most recent call last):
File "/home/ubuntu/www/weiboSpider/weibo_spider/writer/mysql_writer.py", line 72, in _mysql_insert
cursor.executemany(
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 191, in executemany
self.rowcount = sum(self.execute(query, arg) for arg in args)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 191, in
self.rowcount = sum(self.execute(query, arg) for arg in args)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 151, in execute
query = self.mogrify(query, args)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 129, in mogrify
query = query % self._escape_args(args, conn)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 102, in _escape_args
return tuple(conn.literal(arg) for arg in args)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 102, in
return tuple(conn.literal(arg) for arg in args)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/connections.py", line 542, in literal
return self.escape(obj, self.encoders)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/connections.py", line 535, in escape
return converters.escape_item(obj, self.charset, mapping=mapping)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/converters.py", line 23, in escape_item
val = encoder(val, charset, mapping)
File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/converters.py", line 30, in escape_dict
raise TypeError("dict can not be used as parameter")
TypeError: dict can not be used as parameter
1条微博写入MySQL数据库完毕
Progress: 0%| | 0/53904 [00:03<?, ?it/s]
共爬取1条微博
信息抓取完毕


ubuntu @ ubuntu in ~/www/weiboSpider on git:master x [14:29:22]
$ python3 --version
Python 3.10.12

试过mysql5和mysql8都一样是user表能写入,weibo表无法写入

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions