Skip to content

[DNM,Cloud] Cloud master#240

Open
CalvinNeo wants to merge 136 commits intolrz/test-111from
ldz/adapt-raft-engine-uni-ps
Open

[DNM,Cloud] Cloud master#240
CalvinNeo wants to merge 136 commits intolrz/test-111from
ldz/adapt-raft-engine-uni-ps

Conversation

@CalvinNeo
Copy link
Member

@CalvinNeo CalvinNeo commented Dec 21, 2022

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

lidezhu and others added 30 commits November 15, 2022 17:47
@lidezhu
alpha version
1b55bc4
@CalvinNeo
introduce cache mem info
1e978b9 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
partially run
758e22f 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
it can work
2efbf19 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix test bug
c3afc97 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
use snapshot to compose raft apply state
e6228f5 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
do not copy all meta
f529bce 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fmt
9da03d1 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge branch 'raftstore-proxy' into another-fast-add-demo
7a93e5f
@CalvinNeo
fast add peer
a652cbd 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix error router
22afec3 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy' into another-…
2da4c93 
…fast-add-demo
@CalvinNeo
fix merge
0a5c25b 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix logs
1c11747 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
2fb1939 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
fb0917b 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
7da9b92 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
fa6099c 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge fast add peer (#230)
1b335a3
@CalvinNeo
add test for paused
660eb73 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix ci
39f6a14 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix ci
fffc345 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix tests
59e1616 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix tests
eb2c930 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix tests
a9ff447 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
support re add peer
4929749 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
re fmt
e4d6453 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Add support for recover and re-add peer (#231)
391ba67
@CalvinNeo
add tests for apply snapshot
d6617f9 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
add split and merge test
8b7abda 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
Connor1996 and others added 19 commits January 20, 2023 22:43
@Connor1996
batch-system: add priority scheduling for batch system (tikv#14065)
9726e56 
ref tikv#13730

Support priority-based scheduling for the apply batch system.

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@CalvinNeo
[Cloud] Merge raftstore-proxy master(PR #249) (#259)
fcaa14a
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-release-6.6-2
5e32274
@andreid-db
config: allow starting TiKV nodes with <1 CPU (tikv#14084)
c353910 
close tikv#13586, close tikv#13752, ref tikv#14017

Signed-off-by: Andrei Dragus <andrei.dragus@databricks.com>
@HuSharp @ti-chi-bot
pd_client: replace PD_REQUEST_HISTOGRAM_VEC with static metrics (tikv…
68710b9 
…#14087)

close tikv#14086

PD_REQUEST_HISTOGRAM_VEC can be changed to use static metrics to improve performance.

Signed-off-by: husharp <jinhao.hu@pingcap.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@CalvinNeo
adapt tikv's feature
1f04c72 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@LittleFall
copr: (refactor) Lift heap struct out from top_n_executor (tikv#14096)
b1936e6 
ref tikv#13936

Signed-off-by: Zhi Qi <qizhi@pingcap.com>
@JmPotato @ti-chi-bot
raftstore: allow the read request with a smaller ts during flashback (t…
0ce3485 
…ikv#14088)

close tikv#14045

- Store the flashback `start_ts` in region meta.
- Allow the read request with a smaller ts during flashback.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@iosmanthus
import: sst_importer support download SST and rewrite into keyspace d…
7ec73fd 
…ata. (tikv#14046)

ref tikv#12999

import: sst_importer support download SST and rewrite into keyspace data.

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-release-6.6-2
fc18cfa
@tonyxuqqi
rocksdb: reduce rocksdb block size to 16KB (tikv#14053)
ec2f4dc 
close tikv#14052

The writecf and defaultcf's default block size is changed to 16KB to improve read performance (reduce read amplification)

Signed-off-by: qi.xu <tonxuqi@outlook.com>

Co-authored-by: qi.xu <tonxuqi@outlook.com>
@tonyxuqqi @ti-chi-bot
storage: add an alias partitioned-raft-kv for RaftKv2 (tikv#14083)
15d6040 
ref tikv#12842

add an alias partitioned-raft-kv for RaftKv2

Signed-off-by: qi.xu <tonxuqi@outlook.com>

Co-authored-by: qi.xu <tonxuqi@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@CalvinNeo
Merge pull request #262 from CalvinNeo/merge-tikv-release-6.6-2
6444536 
Merge master about 6.6(7ec73fd)
@overvenus @ti-chi-bot
resolved_ts: reduce network traffic by filter regions (tikv#14098)
23a2288 
close tikv#14092

resolved_ts: reduce network traffic by filter regions

Signed-off-by: Neil Shen <overvenus@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@BusyJay @ti-chi-bot
raftstore-v2: fix peer not cleanup when it replicates more logs (tikv…
a33eb2d 
…#14101)

ref tikv#12842

If it accepts more logs than conf remove itself, applied_index == commit_index
will never be true. So we should check if it's a tombstone already first.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@you06
cop: handle unset scan details in store batch (tikv#14102)
9c0df6d 
close tikv#14109

Signed-off-by: you06 <you1474600@gmail.com>
@nolouch
resource_control: unify wru/rru to ru (tikv#14121)
db14c53 
close tikv#14120

resource_control: unify wru/rru to ru

Signed-off-by: nolouch <nolouch@gmail.com>
@CalvinNeo
[Cloud] Add tests for assertion of single replica migration (#264)
df5cc1d 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@HuSharp
pd_client: fix item value type (tikv#14106)
d1d2920 
close tikv#14104

We need to use the new field to support item value as bytes to avoid proto string check failures.

Signed-off-by: husharp <jinhao.hu@pingcap.com>
@CalvinNeo CalvinNeo changed the base branch from raftstore-proxy to lrz/test-111 February 2, 2023 08:36
BusyJay and others added 9 commits February 3, 2023 13:57
@BusyJay
raftstore-v2: add tablet logger and update dep (tikv#14129)
3791560 
ref tikv#12842

- Update raft-engine to fix data corruption during restart
- Add tablet logger so we can know which tablet the logs belongs to

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@CalvinNeo
raftstore: Observe when receive raft message (tikv#14043)
c8c1ca8 
ref tikv#13855

Introduce observers when receive raft message.

Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge remote-tracking branch 'tikv/master' into merge-tikv-6.6-3
17a13b0
@CalvinNeo
adapt to tikv master
31a0fed 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
[Cloud] Fix before merge master (#265)
b96048f
@CalvinNeo
Merge pull request #266 from CalvinNeo/merge-tikv-6.6-3
46ec025 
Merge TiKV master(c8c1ca8)
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy' into merge-mv…
964dbb4 
…p-to-tikv-master
@CalvinNeo
merge tikv upstream/raftstore-proxy
4f4fcbb 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge pull request #267 from CalvinNeo/merge-mvp-to-tikv-master
c1e23d6 
[Cloud] Merge raftstore-proxy with new TiKV observer into cloud branch
@CLAassistant
Copy link

CLAassistant commented Feb 3, 2023 •
edited
Loading

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 16 committers have signed the CLA.

✅ HuSharp
✅ YangKeao
✅ CalvinNeo
✅ LittleFall
❌ tabokie
❌ YuJuncen
❌ andreid-db
❌ ethercflow
❌ Connor1996
❌ iosmanthus
❌ overvenus
❌ BusyJay
❌ JmPotato
❌ tonyxuqqi
❌ nolouch
❌ you06
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.