首页 注册 登录
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
广告
6EyesFlyFish
V2EX  ›  Node.js

express 压缩接口响应的数据量

  •   
  •   6EyesFlyFish · 2021-03-04 16:44:18 +08:00 · 1836 次点击
    这是一个创建于 1396 天前的主题，其中的信息可能已经有所发展或是发生改变。
    express 接口假如响应的数据量达到 7Kb 的时候，在高并发压测的情况下，异常率很高，有没有什么办法压缩数据量？
    响应数据如下：

    {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 22.95,
    "content": "ok",
    "item": [{
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 107.22,
    "nick": "糖衣炮弹 54720",
    "rank": "1",
    "score": "180",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 37.99,
    "nick": "糖衣炮弹 54717",
    "rank": "2",
    "score": "100",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://thirdwx.qlogo.cn/mmopen/vi_32/qDNuYJXiaUZJSlZq2XnmaBDNIIUJBlmPGNZpvbbv6NHibLzIGwzKDmzvibZTd3B0rm7I5BQ0Tcc0aLicU027L7IDibg/132",
    "all_time": 60.41,
    "nick": "一船星辉",
    "rank": "3",
    "score": "120",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 36.1,
    "nick": "54723",
    "rank": "4",
    "score": "90",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 43.72,
    "nick": "54725",
    "rank": "5",
    "score": "90",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 25.32,
    "nick": "54719",
    "rank": "6",
    "score": "70",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 28.51,
    "nick": "54713",
    "rank": "7",
    "score": "70",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 38.81,
    "nick": "54714",
    "rank": "8",
    "score": "80",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 21.8,
    "nick": "54718",
    "rank": "9",
    "score": "60",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 32.19,
    "nick": "54726",
    "rank": "10",
    "score": "70",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 18.34,
    "nick": "54716",
    "rank": "11",
    "score": "50",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 28.7,
    "nick": "54724",
    "rank": "12",
    "score": "60",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 19.37,
    "nick": "54727",
    "rank": "13",
    "score": "50",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 22.95,
    "nick": "54715",
    "rank": "14",
    "score": "50",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 34.49,
    "nick": "糖衣炮弹 oaAq05gIwIhVcT4ydGkqfqaodfdo11",
    "rank": "15",
    "score": "60",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 18.08,
    "nick": "54711",
    "rank": "16",
    "score": "40",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 19.53,
    "nick": "糖衣炮弹 54721",
    "rank": "17",
    "score": "40",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 9.78,
    "nick": "糖衣炮弹 54722",
    "rank": "18",
    "score": "20",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 4.53,
    "nick": "54710",
    "rank": "19",
    "score": "10",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 14.79,
    "nick": "糖衣炮弹",
    "rank": "20",
    "score": "20",
    "wrongcount": "0"
    }, {
    "HeadImgUrl": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd",
    "all_time": 14.74,
    "nick": "54712",
    "rank": "21",
    "score": "10",
    "wrongcount": "0"
    }],
    "all_score": "50",
    "nick": "54715",
    "pNumber": "166556",
    "rank": "14",
    "status": 2000,
    "wrongcount": "0"
    }
  • headimgurl
  • wrongcount
  • all_time
  • Nick
    11 条回复  •  2021-03-06 14:24:29 +08:00
    libook
        1
    libook  
       2021-03-04 16:51:13 +08:00
    异常的具体信息发一下，可能跟数据量大小没关系。
    6EyesFlyFish
        2
    6EyesFlyFish  
    OP
       2021-03-04 17:14:01 +08:00
    异常信息是连接重置
    Response code:Non HTTP response code: java.net.SocketException
    Response message:Non HTTP response message: Connection reset

    我把数据量减少，再压测就没有异常了
    6EyesFlyFish
        3
    6EyesFlyFish  
    OP
       2021-03-04 17:14:37 +08:00
    @libook
    libook
        4
    libook  
       2021-03-04 17:18:07 +08:00
    Node 上、服务器系统级别、反向代理服务器等等，是否异常日志呢？
    以及服务器资源情况，负载、网络连接数、磁盘 IO 、内存使用量等等。
    killergun
        5
    killergun  
       2021-03-04 17:25:01 +08:00
    我感觉这个是不是设置下服务 response 可响应大小就行了
    killergun
        6
    killergun  
       2021-03-04 17:25:39 +08:00
    @killergun 还有就是设置响应超时时间
    6EyesFlyFish
        7
    6EyesFlyFish  
    OP
       2021-03-04 17:41:07 +08:00
    @libook 腾讯云服务器 8 核 16G，0 负载，8%内存使用，ulimit 设置了 100W，没有反向代理，异常日志没有，部分连接直接被重置了
    6EyesFlyFish
        8
    6EyesFlyFish  
    OP
       2021-03-04 17:44:32 +08:00
    @killergun express 可响应大小？在哪设置？我看了官网，似乎没有这方面的介绍
    wakzz
        9
    wakzz  
       2021-03-05 09:28:07 +08:00
    建议楼主抓个包，把异常 TCP 抓到后贴出来，单单通过楼主目前提供的数据推测原因有点困难
    6EyesFlyFish
        10
    6EyesFlyFish  
    OP
       2021-03-05 17:03:04 +08:00
    @wakzz 搞定了，网络带宽问题，数据量太大了
    wakzz
        11
    wakzz  
       2021-03-06 14:24:29 +08:00
    @6EyesFlyFish 这让我想起一次跟第三方联调时压测的网络错误问题，抓包后发现压测到某个阈值就有大量的网络超时，最后查明是对方公司的宽带不够，吐血
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   1012 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 24ms · UTC 18:52 · PVG 02:52 · LAX 10:52 · JFK 13:52
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.