[seata]global_table status=4的数据没有删除

2024-02-02 841 views
8

近几天开始 global里状态是4的数据越来越多,branch和lock表都没有对应的数据,会是哪里的问题? seata1.5.2

回答

5

创建时间和更新时间一致,并且每天以十万+的数量增长,之前的也没删除掉,有两个方法记录最多,其他几十条他俩几万,但是代码没做改动,表结构上前几天加了lock_table 主键和row_key的长度,其他也没啥改动

6

创建时间和更新时间一致,并且每天以十万+的数量增长,之前的也没删除掉,有两个方法记录最多,其他几十条他俩几万,但是代码没做改动,表结构上前几天加了lock_table 主键和row_key的长度,其他也没啥改动

创建时间跟更新时间不可能一致,状态都修改过了,这个xid你自己可以通过日志搜索到到底有没有进行rollback

3

image

7

@lichao5188150 Determine if the problem is a performance issue or a bug. To see if the oldest record is deleted over time, if not to see if the rollback transaction was caused by a dirty write, look for xid analysis through the server log.

2

image xid = '10.182.87.123:8091:36436464783471524'

9

15:59:59.953 INFO --- [verHandlerThread_1_32_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform,transactionServiceGroup: themis-seata-group, transactionName: claimCrab,timeout:60000,xid:10.182.87.123:8091:36436464783471524 15:59:59.957 INFO --- [verHandlerThread_1_40_500] io.seata.server.coordinator.DefaultCore : Rollback global transaction successfully, xid = 10.182.87.123:8091:36436464783471524. 15:59:59.958 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : xid=10.182.87.123:8091:36436464783471524,extraData=null,clientIp:10.181.10.13,vgroup:themis-seata-group 15:59:59.962 INFO --- [verHandlerThread_1_27_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471512, branchId = 36436464783471527, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_complain_action:420437337 15:59:59.963 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471512,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_complain_action:420437337 ,clientIp:10.181.13.13,vgroup:themis-seata-group 15:59:59.968 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : xid=10.182.87.123:8091:36436464783471512,extraData=null,clientIp:10.181.13.13,vgroup:themis-seata-group 15:59:59.974 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_complain:35012205 ,clientIp:10.181.11.16,vgroup:themis-seata-group 15:59:59.976 INFO --- [verHandlerThread_1_21_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471530, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_complain:35012205 16:00:00.026 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3406/themis_flowable,lockKey=act_ru_execution:43e4206b-2153-11ee-ac3e-6267a313cf88,43e4206f-2153-11ee-ac3e-6267a313cf88,43e42071-2153-11ee-ac3e-6267a313cf88;act_ru_variable:43e4206c-2153-11ee-ac3e-6267a313cf88,43e4206d-2153-11ee-ac3e-6267a313cf88,43e4206e-2153-11ee-ac3e-6267a313cf88;act_ru_actinst:43e42070-2153-11ee-ac3e-6267a313cf88,43e42072-2153-11ee-ac3e-6267a313cf88,43e42073-2153-11ee-ac3e-6267a313cf88,43e42074-2153-11ee-ac3e-6267a313cf88,43e42075-2153-11ee-ac3e-6267a313cf88,43e42077-2153-11ee-ac3e-6267a313cf88,43e42078-2153-11ee-ac3e-6267a313cf88;act_ru_task:43e42076-2153-11ee-ac3e-6267a313cf88,43e42079-2153-11ee-ac3e-6267a313cf88 ,clientIp:10.181.12.18,vgroup:themis-seata-group 16:00:00.030 INFO --- [verHandlerThread_1_37_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform-schedule,transactionServiceGroup: themis-seata-group, transactionName: overFlow(com.yunda.model.ThemisAgingCheck),timeout:60000,xid:10.182.87.123:8091:36436464783471533 16:00:00.031 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : timeout=60000,transactionName=overFlow(com.yunda.model.ThemisAgingCheck),clientIp:10.181.21.40,vgroup:themis-seata-group 16:00:00.031 INFO --- [verHandlerThread_1_14_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471532, resourceId = jdbc:mysql://10.181.63.13:3406/themis_flowable ,lockKeys = act_ru_execution:43e4206b-2153-11ee-ac3e-6267a313cf88,43e4206f-2153-11ee-ac3e-6267a313cf88,43e42071-2153-11ee-ac3e-6267a313cf88;act_ru_variable:43e4206c-2153-11ee-ac3e-6267a313cf88,43e4206d-2153-11ee-ac3e-6267a313cf88,43e4206e-2153-11ee-ac3e-6267a313cf88;act_ru_actinst:43e42070-2153-11ee-ac3e-6267a313cf88,43e42072-2153-11ee-ac3e-6267a313cf88,43e42073-2153-11ee-ac3e-6267a313cf88,43e42074-2153-11ee-ac3e-6267a313cf88,43e42075-2153-11ee-ac3e-6267a313cf88,43e42077-2153-11ee-ac3e-6267a313cf88,43e42078-2153-11ee-ac3e-6267a313cf88;act_ru_task:43e42076-2153-11ee-ac3e-6267a313cf88,43e42079-2153-11ee-ac3e-6267a313cf88 16:00:00.045 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : timeout=60000,transactionName=overFlow(com.yunda.model.ThemisAgingCheck),clientIp:10.181.10.15,vgroup:themis-seata-group 16:00:00.045 INFO --- [verHandlerThread_1_42_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform-schedule,transactionServiceGroup: themis-seata-group, transactionName: overFlow(com.yunda.model.ThemisAgingCheck),timeout:60000,xid:10.182.87.123:8091:36436464783471534 16:00:00.052 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_complain_process:135583495 ,clientIp:10.181.11.16,vgroup:themis-seata-group 16:00:00.053 INFO --- [verHandlerThread_1_23_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform,transactionServiceGroup: themis-seata-group, transactionName: handleOrder,timeout:60000,xid:10.182.87.123:8091:36436464783471535 16:00:00.054 INFO --- [verHandlerThread_1_24_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471537, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_complain_process:135583495 16:00:00.057 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : timeout=60000,transactionName=handleOrder,clientIp:10.181.10.17,vgroup:themis-seata-group 16:00:00.063 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_complain_action:420437338 ,clientIp:10.181.11.16,vgroup:themis-seata-group 16:00:00.065 INFO --- [verHandlerThread_1_38_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471539, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_complain_action:420437338 16:00:00.075 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471533,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_aging_check:98640176 ,clientIp:10.181.21.40,vgroup:themis-seata-group 16:00:00.076 INFO --- [verHandlerThread_1_10_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471533, branchId = 36436464783471541, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_aging_check:98640176 16:00:00.081 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_claim_detail:2385734 ,clientIp:10.181.11.16,vgroup:themis-seata-group 16:00:00.083 INFO --- [verHandlerThread_1_15_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471543, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_claim_detail:2385734 16:00:00.086 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471534,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_aging_check:98931987 ,clientIp:10.181.10.15,vgroup:themis-seata-group 16:00:00.088 INFO --- [verHandlerThread_1_46_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471534, branchId = 36436464783471545, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_aging_check:98931987 16:00:00.097 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471535,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_complain:35009201 ,clientIp:10.181.10.17,vgroup:themis-seata-group 16:00:00.098 INFO --- [verHandlerThread_1_47_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471535, branchId = 36436464783471547, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_complain:35009201 16:00:00.109 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471534,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_reminder:45329852 ,clientIp:10.181.10.15,vgroup:themis-seata-group 16:00:00.112 INFO --- [verHandlerThread_1_48_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471534, branchId = 36436464783471549, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_reminder:45329852 16:00:00.128 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471521,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3406/themis_flowable,lockKey=act_ru_actinst:43f47340-2153-11ee-81f8-da5781d893f5,43f49a51-2153-11ee-81f8-da5781d893f5,43f49a52-2153-11ee-81f8-da5781d893f5,43f49a53-2153-11ee-81f8-da5781d893f5;act_ru_actinst:f907b8a2-1fd2-11ee-8d0c-5e40c78b7d9e;act_ru_task:f907b8a3-1fd2-11ee-8d0c-5e40c78b7d9e;act_ru_task:43f49a54-2153-11ee-81f8-da5781d893f5;act_ru_execution:5acc3684-1d61-11ee-8838-8ac454ff56a0;act_ru_variable:ef57c217-1e10-11ee-8d0c-5e40c78b7d9e ,clientIp:10.181.10.16,vgroup:themis-seata-group 16:00:00.132 INFO --- [verHandlerThread_1_26_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471521, branchId = 36436464783471551, resourceId = jdbc:mysql://10.181.63.13:3406/themis_flowable ,lockKeys = act_ru_actinst:43f47340-2153-11ee-81f8-da5781d893f5,43f49a51-2153-11ee-81f8-da5781d893f5,43f49a52-2153-11ee-81f8-da5781d893f5,43f49a53-2153-11ee-81f8-da5781d893f5;act_ru_actinst:f907b8a2-1fd2-11ee-8d0c-5e40c78b7d9e;act_ru_task:f907b8a3-1fd2-11ee-8d0c-5e40c78b7d9e;act_ru_task:43f49a54-2153-11ee-81f8-da5781d893f5;act_ru_execution:5acc3684-1d61-11ee-8838-8ac454ff56a0;act_ru_variable:ef57c217-1e10-11ee-8d0c-5e40c78b7d9e 16:00:00.140 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471535,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3406/themis_flowable,lockKey=act_ru_task:43f64841-2153-11ee-8a3e-f268bb5ca17e;act_ru_actinst:e4c09286-2150-11ee-bda4-c645caa9fcee;act_ru_task:e4c09287-2150-11ee-bda4-c645caa9fcee;act_ru_execution:95716186-2150-11ee-81f8-da5781d893f5;act_ru_variable:43f5d30c-2153-11ee-8a3e-f268bb5ca17e;act_ru_actinst:43f6212d-2153-11ee-8a3e-f268bb5ca17e,43f6212e-2153-11ee-8a3e-f268bb5ca17e,43f6212f-2153-11ee-8a3e-f268bb5ca17e,43f64840-2153-11ee-8a3e-f268bb5ca17e ,clientIp:10.181.14.13,vgroup:themis-seata-group 16:00:00.150 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : SeataMergeMessage xid=10.182.87.123:8091:36436464783471533,branchType=AT,resourceId=jdbc:mysql://10.181.63.13:3404/themis_center,lockKey=themis_reminder:45188720 ,clientIp:10.181.21.40,vgroup:themis-seata-group 16:00:00.152 INFO --- [rverHandlerThread_1_2_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471533, branchId = 36436464783471555, resourceId = jdbc:mysql://10.181.63.13:3404/themis_center ,lockKeys = themis_reminder:45188720 16:00:00.153 INFO --- [verHandlerThread_1_29_500] i.seata.server.coordinator.AbstractCore : Register branch successfully, xid = 10.182.87.123:8091:36436464783471535, branchId = 36436464783471553, resourceId = jdbc:mysql://10.181.63.13:3406/themis_flowable ,lockKeys = act_ru_task:43f64841-2153-11ee-8a3e-f268bb5ca17e;act_ru_actinst:e4c09286-2150-11ee-bda4-c645caa9fcee;act_ru_task:e4c09287-2150-11ee-bda4-c645caa9fcee;act_ru_execution:95716186-2150-11ee-81f8-da5781d893f5;act_ru_variable:43f5d30c-2153-11ee-8a3e-f268bb5ca17e;act_ru_actinst:43f6212d-2153-11ee-8a3e-f268bb5ca17e,43f6212e-2153-11ee-8a3e-f268bb5ca17e,43f6212f-2153-11ee-8a3e-f268bb5ca17e,43f64840-2153-11ee-8a3e-f268bb5ca17e 16:00:00.177 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : timeout=60000,transactionName=qualityCheck,clientIp:10.181.10.13,vgroup:themis-seata-group 16:00:00.178 INFO --- [verHandlerThread_1_30_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform-schedule,transactionServiceGroup: themis-seata-group, transactionName: qualityCheck,timeout:60000,xid:10.182.87.123:8091:36436464783471556 16:00:00.186 INFO --- [ batchLoggerPrint_1_1] i.s.c.r.p.server.BatchLogHandler : timeout=60000,transactionName=pcAutoStatement,clientIp:10.181.11.11,vgroup:themis-seata-group 16:00:00.186 INFO --- [verHandlerThread_1_31_500] i.s.s.coordinator.DefaultCoordinator : Begin new global transaction applicationId: themis-sharing-platform-schedule,transactionServiceGroup: themis-seata-group, transactionName: pcAutoStatement,timeout:60000,xid:10.182.87.123:8091:36436464783471557 看下这个日志 回滚成功没看到异常

8

image

时间精度存在问题,可以加一下精度,否则看不到毫秒单位

5

还有一种可能就是你的并发量太大,导致删除速度远不及新增的数据,你锁定当前查出来最早的那条xid,过一段时间查看还在不在

3

没有branch和lock,跟脏写应该是无关的,怀疑是请求量太大,回滚事务的新增远高于100的limit

3

jstack看下RetryRollbacking这个线程是否能找到,是否在工作

9

server.retryDeadThreshold 排查下这个配置项是否被你们使用(从你们使用的配置中心中寻找),如果使用请看下值是多少,默认应该是130000,130秒后删除,如果你们配置的非常大,没有删除也就合理了

3

server.retryDeadThreshold 没有配置这个 肯定是默认值的,如果说是数据量大的话,我不太确定 因为11号之前搜不到这种数据,11号开始关注到,3天了还没删掉

6
91e433f906731600aedfab550e48e7b9
8

堆栈一起晒出来