MatrixOne-v1.2.1

Release date: June 30, 2024
MatrixOne version: v1.2.1

Compared with the previous v1.2.0, v1.2.1 doesn't introduce new features but only focuses on bug fixes and minor optimizations.

Improvements

• Memory cache supports multiple memory allocators.
• Optimize the performance of show accounts.
• Optimize observability metrics for fileservice.
• Support incremental backup.
• Optimize TCP packet estimation.
• Refactor mologging.
• Support restoration of system tables.
• Support manual merge based on zonemap.

Bugfix

Fixed bugs related to snapshot reads, MySQL compatibility, lockservice, runtime filters, and system table upgrades. Check the What's Changed section for details.

What's Changed

• fix query result by @daviszhen in #16266
• fix restore partition table (#1.2-dev) by @YANGGMM in #16256
• pick #16245 to 1.2 : reduce agg context memory usage. by @m-schen in #16253
• sync to 1.2: optimize show accounts by @gouhongshen in #16241
• modify snapshot case syntax(1.2-dev) by @Ariznawlll in #16233
• fix visit tombstone, cherry-pick #16163 by @jiangxinmeng1 in #16189
• add debug info & profile interval by @daviszhen in #16223
• move some logic from compile to run (#15983) by @ouyuanning in #16079
• Fix mo cloud-issue-3298 (#16254) by @triump2020 in #16281
• [fix] use runtime filter for multi-row prepared insert stmt (cherry-pick to 1.2-dev) by @aunjgr in #16261
• refactor tn handler 1.2 by @aptend in #16136
• Support not deleting the checkpoint referenced by the snapshot 1.2 by @LeftHandCold in #16128
• test restore fk table by @YANGGMM in #16259
• Extended version upgrade package (#16049) -1.2-dev by @qingxinhome in #16271
• handle alter table add column issue (#16122) -1.2-dev by @qingxinhome in #16267
• inc ref of new bind in restart cn by @iamlinjunhong in #16294
• fix TestDedup2, cherry-pick #16301 by @jiangxinmeng1 in #16302
• syn to 1.2: collecting the mo_database, mo_tables, mo_columns and cluster table size for non-sys account. by @gouhongshen in #16287
• GC needs to be compatible with the earliest checkpoint for 1.2 by @LeftHandCold in #16305
• GC tombstone object (#16102) 1.2 by @LeftHandCold in #16238
• Refactor: change Chan of WaitRegister from *Batch to *Message(contains batch & error) by @ouyuanning in #16263
• change keep_user_target_list_in_result default to 1 by @YANGGMM in #16234
• Improve sql usage 1.2 dev by @w-zr in #16277
• fix 11250 on 1.2-dev by @w-zr in #16327
• pick 16239 and 16248 to 1.2-dev by @XuPeng-SH in #16257
• convert dup error if txn is orphan to 1.2 by @zhangxu19830126 in #16334
• fix TestRemovedCronTask panic on 1.2-dev by @w-zr in #16323
• fix restore view table (#1.2-dev) by @YANGGMM in #16330
• fix can not connect to remote backend when down cn by @iamlinjunhong in #16329
• fileservice: add i/o timeout as retryable error by @reusee in #16347
• add case for restore view(1.2-dev) by @Ariznawlll in #16335
• update save profile by @daviszhen in #16356
• fix show create table with fk in other database (#1.2-dev) by @YANGGMM in #16349
• add some recover logs by @volgariver6 in #16363
• fix rollback create table by @jiangxinmeng1 in #16369
• fix does not unlock deadlock txn by @iamlinjunhong in #16382
• fileservice: more metrics for Read by @reusee in #16390
• rm tag 16311 16306 16297(1.2-dev) by @Ariznawlll in #16396
• sync to 1.2: remove the old fast ranges and update metrics by @gouhongshen in #16359
• fix force flush timeout by @jiangxinmeng1 in #16381
• [bug] proxy: do not add to queue multi times when scaling CN by @volgariver6 in #16388
• add txn id by @daviszhen in #16395
• remove NullComputationWrapper by @daviszhen in #16290
• [cherry-pick-16195] : remove lock op for secondary index by @jensenojs in #16409
• Fix GetByteSlice out range for 1.2 by @LeftHandCold in #16418
• Optimize the estimation of tcp packet num send back to client in mo1.2 by @xzxiong in #16412
• fix panic during merge on 1.2-dev by @w-zr in #16416
• [opt] merge filters on composite primary keys in plan (cp to 1.2-dev) by @aunjgr in #16406
• fix 15848, cherry-pick #16435 by @jiangxinmeng1 in #16436
• sync to 1.2, re-pr: remove the old fast ranges and update metrics by @gouhongshen in #16444
• Fix fifocache evict for 1.2 by @LeftHandCold in #16451
• refactor mo logging for rawlog and statement_info in mo1.2 by @xzxiong in #16411
• [Cherry-pick] fix snapshot restore fk cross db by @ck89119 in #16454
• Optimize QueryResult decoupling from motrace in mo1.2 by @xzxiong in #16426
• cherry_pick-Supports Information_Schema FILES and is compatible with MySQL (#15… by @qingxinhome in #16457
• [cherry-pick-16445] : use ctx instead of proc.ctx for rangesOnePart by @jensenojs in #16465
• Fix-RW-Conflict by @triump2020 in #16477
• fix txnid by @daviszhen in #16458
• [cherry-pick-16424] : add log to track 16273 by @jensenojs in #16472
• chore: stable bvt/result_count.sql result check and enable load_data_LOCAL_infile case in mo1.2 by @xzxiong in #16478
• pick pr #16452 to 1.2-dev, fix a bug will cause hung if error happens. by @m-schen in #16453
• Support upgrading to skip Genesis version functionality-1.2-dev by @qingxinhome in #16486
• delay Ranges for partitioned table-1.2-dev by @qingxinhome in #16268
• fix stats misuse in some cases to 1.2-dev by @badboynt1 in #16500
• [CherryPick] : Fix failed BVT query in Secondary Index (#16429) by @arjunsk in #16499
• add test for restore view table and fk table (#1.2-dev) by @YANGGMM in #16432
• sync to 1.2: supporting reader linear search on unsorted fake pk col by @gouhongshen in #16461
• add hidden force option for objprune 1.2 by @aptend in #16514
• support restore system table (#1.2-dev) by @YANGGMM in #16462
• sync to 1.2: skip collecting mo_tables, mo_databases, mo_columns and cluster tables for non-sys. by @gouhongshen in #16515
• chore: make sure no aggr in bvt result_count in 1.2 by @xzxiong in #16536
• remove log print from automaxprocs to 1.2-dev by @badboynt1 in #16547
• malloc: fix log level by @reusee in #16509
• fix lock service ut by @iamlinjunhong in #16518
• support merge objects by table id on 1.2-dev by @w-zr in #16520
• sync to 1.2: making collect dirty blocks lazily by @gouhongshen in #16527
• chore: refactor purge_log calling objprune if target_date + 24h <= now (mo1.2) by @xzxiong in #16550
• sync to 1.2: filter optimization of the loadDeletes by @gouhongshen in #16555
• [enhancement] logtail: change the default timeout of sending resp. by @volgariver6 in #16561
• [enhancement] proxy: filter CNs that are not in working state. by @volgariver6 in #16559
• metric: fix fileservice board by @reusee in #16554
• add issue 16178 cases by @heni02 in #16545
• fix drop publication if exists statement (#1.2-dev) by @YANGGMM in #16464
• add case for restore system table(1.2-dev) by @Ariznawlll in #16481
• clean code by @daviszhen in #16571
• rewrite 'insert ... on duplicate key update' to 'replace into' in some case by @ouyuanning in #16568
• append log for upgrade and sqlExecutoer1.2 dev by @qingxinhome in #16576
• Bvt if disable trace in 1.2 by @xzxiong in #16582
• pick #16523 to 1.2-dev by @m-schen in #16586
• add bvt tag(#1.2-dev) by @YANGGMM in #16590
• fix a remote run bug, rpc stream should be closed after pipeline cleanup. by @m-schen in #16588
• cherry-pick fix bvt test by @YANGGMM in #16579
• fix snapshot name ambiguous (#1.2-dev) by @YANGGMM in #16580
• GC needs to consume all the mo_snapshot tables for 1.2 by @LeftHandCold in #16591
• [fix] : fix moc3399 by @jensenojs in #16594
• rmTag15901(1.2-dev) by @Ariznawlll in #16592
• remove log print from automaxprocs to 1.2-dev by @badboynt1 in #16599
• fix convert string to timestamp by @ck89119 in #16621
• support restore publication db (#1.2-dev) by @YANGGMM in #16488
• Fix condition to ignore delete booking if no transfer is needed on 1.2-dev by @w-zr in #16632
• [Cherry-pick] refactor system variables to support account isolation by @ck89119 in #16636
• add case for restore pub_sub(1.2-dev) by @Ariznawlll in #16603
• add debug info for panic by @daviszhen in #16635
• Optimize logs by @qingxinhome in #16647
• Support merge policy parameter 1.2 dev by @w-zr in #16669
• handle null in convertRowsIntoBatch by @ck89119 in #16675
• Fix enum type system variable check by @ck89119 in #16692
• [opt] merge filters on composite primary keys in plan (1.2-dev) by @aunjgr in #16693
• fix restore pub db (#1.2-dev) by @YANGGMM in #16668
• sync to 1.2: block reader supports between filter by @gouhongshen in #16687
• cherry-pick HTTP client and DNS changes to 1.2-dev by @reusee in #16718
• rmTag16601_16597(1.2-dev) by @Ariznawlll in #16703
• add global system variable and session variable account isolation cases by @heni02 in #16695
• Handle Cancel Restore Statement Fail by @qingxinhome in #16736
• add issue 16613 cases by @heni02 in #16721
• skip stats for create view to 1.2-dev by @badboynt1 in #16729
• change LIMIT, OFFSET's data type from int64 to uint64 by @daviszhen in #16698
• make add txn error trace async by @zhangxu19830126 in #16758
• add issue 16139 cases by @heni02 in #16734
• add a code owner for 1.2 by @XuPeng-SH in #16763
• sync to 1.2: refactor the block reader filter to support more expressions. by @gouhongshen in #16755
• [bug] launch: fix timeout mechanism when TN service startup by @volgariver6 in #16761
• fileservice: fix fd leak in LocalFS.read by @reusee in #16749
• [1.2-dev] fix: respsect filterFn in pluginRouter by @aylei in #16689
• handle Restore Duplicate Entry-1.2-dev by @qingxinhome in #16743
• Cherry-pick-for-snapshot-read-bug by @triump2020 in #16777
• add cases by @heni02 in #16783
• [opt] retain IN expression in prepared stmt (cp to 1.2-dev) by @aunjgr in #16800
• sync to 1.2: workaround for get col type panic when constructing block reader filter. by @gouhongshen in #16804
• do not rewrite on duplicate key stmt to replace for prepare by @ouyuanning in #16793
• add case for issue3302(1.2-dev). by @Ariznawlll in #16792
• fix panic by @daviszhen in #16776
• Refactor view scope execute-1.2-dev by @qingxinhome in #16787
• Do not apply delete when aobject merge on 1.2 dev by @w-zr in #16789
• add SERVER_MORE_RESULTS_EXISTS judgment by @CJKkkk-315 in #16781
• [CherryPick]: Unhex/ Hex Feature Parity with MySQL by @arjunsk in #16809
• fix a bug about send notify message to remote node. by @m-schen in #16696
• rm tag 16710(1.2-dev) by @Ariznawlll in #16814
• add error to alloc by @daviszhen in #16799
• add deug log by @jiangxinmeng1 in #16820
• change some UnsafeGetStringAt to GetStringAt by @ouyuanning in #16821
• add tag 16709(1.2-dev) by @Ariznawlll in #16818
• sync to 1.2: fix sysbench random points ranges slow by @gouhongshen in #16827
• [bug] clusterservice: add lock for refresh() to protect cn services. by @volgariver6 in #16840
• add query result count by @daviszhen in #16837
• [Cherry-pick] Refactor system variable lower_case_table_names by @ck89119 in #16823
• flush object dedup 1.2 by @aptend in #16860
• change stats update threshold from 1000 to blocknumber to 1.2-dev by @badboynt1 in #16848
• Always push isnull to block filters to 1.2-dev by @badboynt1 in #16852
• rollback the dedup filters to workaround an execution bug by @aunjgr in #16843
• fix validate timeout lock table by @iamlinjunhong in #16862
• add tag 16709(1.2-dev). by @Ariznawlll in #16833
• [Cherry-pick] case insensitive sys db and table names by @ck89119 in #16863
• trace slow flush 1.2 by @aptend in #16876
• fix: check protocol quit before CalculateOutTrafficBytes in 1.2 by @xzxiong in #16865
• Cherry-pick-for-fix-dup-and-inconsistency by @triump2020 in #16874
• update debug log, print appendnodes by @jiangxinmeng1 in #16890
• [Cherry-pick] support set sys var from toml file by @ck89119 in #16891
• Update txn print by @XuPeng-SH in #16894
• fix dump panic(#1.2-dev) by @YANGGMM in #16903
• fileservice: refine disk cache error handling codes by @reusee in #16901
• print file name for sync error by @LeftHandCold in #16902
• constant fold for more expr(1.2) by @ouyuanning in #16908
• fix show create table (#1.2-dev) by @YANGGMM in #16910
• fix show fk by @daviszhen in #16914
• sync to 1.2: refactor in memory reader filter and simplify reader filters. by @gouhongshen in #16915
• [enhancement] logtail: make table subscription async by @volgariver6 in #16887
• change multi clone snapshot op in one op (#1.2-dev) by @YANGGMM in #16813
• file service: tune http read write timeout by @reusee in #16896
• fix desc, show columns from by @daviszhen in #16859
• fix memory related by @daviszhen in #16920
• [fix]: Add log 3421 by @jensenojs in #16907
• Remove list checkpoint meta file (#16723) by @LeftHandCold in #16918
• Fix-OOM: CN will flush inserts into S3 when its count exceeds thresholds by @triump2020 in #16921
• Fix Visit object meta by @LeftHandCold in #16927
• load columnlist by @huby2358 in #16935
• fix slow flush 1.2 by @aptend in #16937
• add bvt test (#1.2-dev) by @YANGGMM in #16945
• change transfer page ttl from 90s to 60s by @XuPeng-SH in #16952
• fix: load data in file with column list error by @huby2358 in #16949
• Modify the string of DECIMAL64/128/256 to match the acceptable string when parsing SQL by @CJKkkk-315 in #16911
• fix check invalid binds by @iamlinjunhong in #16958
• malloc: port new allocator to 1.2 by @reusee in #16670
• sync to 1.2: adding TN deduplication latency metrics. by @gouhongshen in #16934
• sync to 1.2: fix block reader filter case of pk in () and xxx in (). by @gouhongshen in #16961
• fix migrate txn leak & profile name by @daviszhen in #16946
• [bug] trace: change the timeout from 30s to 5m by @volgariver6 in #16973
• refactor the pipeline's remote run by @m-schen in #16923
• Fix Rebuild snapshot's table meta by @LeftHandCold in #16968
• [bug] txn: do not fatal if the waiter is paused when waiting for the ts. by @volgariver6 in #16985
• clean batch by @daviszhen in #16978
• fix truncate by @jiangxinmeng1 in #16989
• Fix error refers to table meta by @LeftHandCold in #16998
• fix incorrect use of connection ctx during remote run. by @m-schen in #17002
• add log to debug lock table created multi times by @zhangxu19830126 in #17010
• fix a bug that caused the remote-run context to lose a deadline by @m-schen in #17017
• Refactor UnresolvedName by @ck89119 in #17000
• fix snapshot-read bug by @triump2020 in #17013
• remove an unnecessary code, modify a log level. by @m-schen in #17033
• retry allocate if orphan txn by @zhangxu19830126 in #17037
• adding log debug size() by @gouhongshen in #17032
• [Cherry-pick] Fix nil pointer in RoutineManager.Closed() by @ck89119 in #17036
• adjust the timeout for orphan transactions by @zhangxu19830126 in #17045
• sync to 1.2: change all filter related "or" expr from binary function to multi function by @gouhongshen in #16991
• add system, busy monitor, by @zhangxu19830126 in #17048
• [bug] filter out the CNs with different commit ID. by @volgariver6 in #17004
• sync to 1.2: adding logic for prefix_in in case vector items are not unique by @gouhongshen in #17042
• debug load local by @daviszhen in #17047
• enable & disable checkpoint by @jiangxinmeng1 in #17043
• Fix a bug in some cases that @m-schen would not call RPC close in #17054
• cherry pick stats related fix to 1.2-dev by @badboynt1 in #17052
• [bug] stats: fix the leak of goroutine by @volgariver6 in #17072
• fix a remoterun bug never close connection. by @m-schen in #17079
• Fix memory leak for partition state by @triump2020 in #17068
• malloc, fileservice: port main branch changes to 1.2-dev by @reusee in #17065
• mo-service: data dir compatibility fixes by @reusee in #17063
• preallocate transfer page by @w-zr in #17058
• add a metric for the pipeline remote run. by @m-schen in #17085
• [enhancement] logtail: change the default send timeout, add logs by @volgariver6 in #17060
• fix lock table move failed by @iamlinjunhong in #17091
• Fix rightjoin.sendlast hung by @zengyan1 in #17038
• fix RPC metrics by @zhangxu19830126 in #17090
• Add handleDiskCleaner for mo_ctl by @LeftHandCold in #17083
• fix get role name of sys tables by @ck89119 in #17102
• fileservice: use noop retryer in aws sdk by @reusee in #17096
• [cherry-pick-16310]: mergeDelete on the hidden table should not update affectedRows by @jensenojs in #17117
• Fix rightjoin.Free hung by @zengyan1 in #17107
• fix delete page error and some typos by @w-zr in #17101
• cherrypick: fix ut failed about race by @daviszhen in #17108
• Init snapshot table must consume a global checkpoint by @LeftHandCold in #17120
• fix memory leak in flushAObjsForSnapshot by @w-zr in #17123
• fix can restart cn when tn crashes by @iamlinjunhong in #17114
• Set transfer page limit 1.2 by @aptend in #17078
• add metric by @daviszhen in #17126
• Cherry pick fix[ ww/dup/datalost] bugs when unsubscribe table by @triump2020 in #17099
• Handle information schema tables issue 1.2 dev by @qingxinhome in #17119
• Fix ww/dup- caused by lost deletes in rangesOnePart by @triump2020 in #17138
• sync to 1.2: in-memory reader filter workaround for special tables. by @gouhongshen in #17137
• sync to 1.2: fix Size() duplicate counting rows size. by @gouhongshen in #17147
• Fix panic by @triump2020 in #17162
• fileservice: make TLS handshake timeout retryable error by @reusee in #17170
• change metadata cache size and policy by @XuPeng-SH in #17187
• cherry pick to 1.2-dev '[Bug]: random points qps in daily regression degrade from 20000+ to 7…' by @badboynt1 in #17191
• Fix dead lock by @triump2020 in #17194
• Fix force merge big data bug&objectIO add object size limit by @LeftHandCold in #17196
• fix bug: do not throw error in remote run by @ouyuanning in #17197
• 1.2 support compile service by @m-schen in #17209

Full Changelog: v1.2.0...v1.2.1