[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-deep-evm-28-gao-tunlv-shuju-guanxian-piliang-charu-copy":3},{"article":4,"author":59},{"id":5,"category_id":6,"title":7,"slug":8,"excerpt":9,"content_md":10,"content_html":11,"locale":12,"author_id":13,"published":14,"published_at":15,"meta_title":16,"meta_description":17,"focus_keyword":18,"og_image":19,"canonical_url":19,"robots_meta":20,"created_at":15,"updated_at":15,"tags":21,"category_name":24,"related_articles":39},"d3000000-0000-0000-0000-000000000128","a0000000-0000-0000-0000-000000000035","Deep EVM #28：高吞吐数据管线——批量插入、COPY和冲突解决","deep-evm-28-gao-tunlv-shuju-guanxian-piliang-charu-copy","构建高吞吐PostgreSQL数据管线：批量插入优化、COPY协议、ON CONFLICT冲突解决和连接池管理。","## 批量插入优化\n\n单条INSERT效率低下。批量插入使用单个语句插入多行：\n\n```sql\nINSERT INTO events (type, data, created_at) VALUES\n  ('block', $1, NOW()),\n  ('block', $2, NOW()),\n  -- ... 数千行\n;\n```\n\n## COPY协议\n\nPostgreSQL的COPY是最快的数据加载方式，比批量INSERT快5-10倍：\n\n```rust\nuse tokio_postgres::types::ToSql;\n\nlet copy_stmt = client\n    .prepare(\"COPY events (type, data, created_at) FROM STDIN BINARY\")\n    .await?;\n\nlet sink = client.copy_in(&copy_stmt).await?;\n\u002F\u002F 写入二进制数据...\n```\n\n## ON CONFLICT冲突解决\n\n```sql\nINSERT INTO prices (pair, price, updated_at)\nVALUES ('ETH\u002FUSDC', 3500.00, NOW())\nON CONFLICT (pair) DO UPDATE SET\n    price = EXCLUDED.price,\n    updated_at = EXCLUDED.updated_at;\n```\n\n## 连接池\n\n使用bb8或deadpool管理连接池：\n\n```rust\nlet pool = bb8::Pool::builder()\n    .max_size(20)\n    .build(manager)\n    .await?;\n```\n\n## 性能基准\n\n| 方法 | 每秒行数 |\n|------|----------|\n| 单条INSERT | ~1,000 |\n| 批量INSERT (1000行) | ~50,000 |\n| COPY | ~200,000 |\n\n## 总结\n\n高吞吐数据管线的关键是使用正确的插入方法。COPY协议提供最高性能，结合连接池和冲突解决策略可以构建可靠的数据管线。","\u003Ch2 id=\"\">批量插入优化\u003C\u002Fh2>\n\u003Cp>单条INSERT效率低下。批量插入使用单个语句插入多行：\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-sql\">INSERT INTO events (type, data, created_at) VALUES\n  ('block', $1, NOW()),\n  ('block', $2, NOW()),\n  -- ... 数千行\n;\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2 id=\"copy\">COPY协议\u003C\u002Fh2>\n\u003Cp>PostgreSQL的COPY是最快的数据加载方式，比批量INSERT快5-10倍：\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-rust\">use tokio_postgres::types::ToSql;\n\nlet copy_stmt = client\n    .prepare(\"COPY events (type, data, created_at) FROM STDIN BINARY\")\n    .await?;\n\nlet sink = client.copy_in(&copy_stmt).await?;\n\u002F\u002F 写入二进制数据...\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2 id=\"on-conflict\">ON CONFLICT冲突解决\u003C\u002Fh2>\n\u003Cpre>\u003Ccode class=\"language-sql\">INSERT INTO prices (pair, price, updated_at)\nVALUES ('ETH\u002FUSDC', 3500.00, NOW())\nON CONFLICT (pair) DO UPDATE SET\n    price = EXCLUDED.price,\n    updated_at = EXCLUDED.updated_at;\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2 id=\"\">连接池\u003C\u002Fh2>\n\u003Cp>使用bb8或deadpool管理连接池：\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-rust\">let pool = bb8::Pool::builder()\n    .max_size(20)\n    .build(manager)\n    .await?;\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2 id=\"\">性能基准\u003C\u002Fh2>\n\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>方法\u003C\u002Fth>\u003Cth>每秒行数\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\n\u003Ctr>\u003Ctd>单条INSERT\u003C\u002Ftd>\u003Ctd>~1,000\u003C\u002Ftd>\u003C\u002Ftr>\n\u003Ctr>\u003Ctd>批量INSERT (1000行)\u003C\u002Ftd>\u003Ctd>~50,000\u003C\u002Ftd>\u003C\u002Ftr>\n\u003Ctr>\u003Ctd>COPY\u003C\u002Ftd>\u003Ctd>~200,000\u003C\u002Ftd>\u003C\u002Ftr>\n\u003C\u002Ftbody>\u003C\u002Ftable>\n\u003Ch2 id=\"\">总结\u003C\u002Fh2>\n\u003Cp>高吞吐数据管线的关键是使用正确的插入方法。COPY协议提供最高性能，结合连接池和冲突解决策略可以构建可靠的数据管线。\u003C\u002Fp>\n","zh","b0000000-0000-0000-0000-000000000001",true,"2026-03-28T10:44:26.325426Z","高吞吐数据管线——批量插入、COPY和冲突解决","PostgreSQL高吞吐数据管线：批量插入、COPY协议、ON CONFLICT冲突解决和连接池管理。","postgresql高吞吐数据管线",null,"index, follow",[22,27,31,35],{"id":23,"name":24,"slug":25,"created_at":26},"c0000000-0000-0000-0000-000000000012","DevOps","devops","2026-03-28T10:44:21.513630Z",{"id":28,"name":29,"slug":30,"created_at":26},"c0000000-0000-0000-0000-000000000022","Performance","performance",{"id":32,"name":33,"slug":34,"created_at":26},"c0000000-0000-0000-0000-000000000005","PostgreSQL","postgresql",{"id":36,"name":37,"slug":38,"created_at":26},"c0000000-0000-0000-0000-000000000001","Rust","rust",[40,47,53],{"id":41,"title":42,"slug":43,"excerpt":44,"locale":12,"category_name":45,"published_at":46},"d0000000-0000-0000-0000-000000000668","为什么Bali在2026年正在成为东南亚的影响力科技中心","weishenme-bali-2026-zhengzai-chengwei-dongnanya-yingxiangli-keji-zhongxin","Bali在东南亚创业生态系统中排名第16位。随着Web3构建者、AI可持续发展初创公司和生态旅游科技公司的集中，该岛正在打造区域影响力科技之都的独特定位。","工程","2026-03-28T10:44:48.898750Z",{"id":48,"title":49,"slug":50,"excerpt":51,"locale":12,"category_name":45,"published_at":52},"d0000000-0000-0000-0000-000000000667","ASEAN数据保护拼图：开发者合规清单","asean-shuju-baohu-pintu-kaifazhe-heguiqingdan","七个ASEAN国家现已拥有全面的数据保护法律，各自具有不同的同意模型、本地化要求和处罚结构。这是一份为构建多国应用程序的开发者准备的实用合规清单。","2026-03-28T10:44:48.893467Z",{"id":54,"title":55,"slug":56,"excerpt":57,"locale":12,"category_name":45,"published_at":58},"d0000000-0000-0000-0000-000000000666","Indonesia 290亿美元数字化转型：软件公司的机遇","indonesia-290yi-meiyuan-shuzihua-zhuanxing-ruanjian-gongsi-jiyu","Indonesia IT服务市场预计在2026年达到290.3亿美元，高于2025年的243.7亿美元。云基础设施、AI、电子商务和数据中心正在推动东南亚最快的增长。","2026-03-28T10:44:48.875457Z",{"id":13,"name":60,"slug":61,"bio":62,"photo_url":19,"linkedin":19,"role":63,"created_at":64,"updated_at":64},"Open Soft Team","open-soft-team","The engineering team at Open Soft, building premium software solutions from Bali, Indonesia.","Engineering Team","2026-03-28T08:31:22.226811Z"]