Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mempool] Use Stream instead of Build + Prefetch Mempool Txs/State + Batch IsRepeat #350

Merged
merged 18 commits into from
Aug 13, 2023

Conversation

patrick-ogrady
Copy link
Contributor

@patrick-ogrady patrick-ogrady commented Aug 12, 2023

Resolves: #249

Testing (10k accounts, 500k txs, M2 Max)

Before [main] (~86856 TPS)

[1] block generation {"t": "5.646185875s", "avg(ms)": 115, "tps": 88555.35596408256}
[2] block generation {"t": "5.6602125s", "avg(ms)": 113, "tps": 88335.90611659898}
[3] block generation {"t": "5.770886625s", "avg(ms)": 117, "tps": 86641.7991706777}
[4] block generation {"t": "5.646529958s", "avg(ms)": 115, "tps": 88549.95966002098}
[5] block generation {"t": "6.082898417s", "avg(ms)": 124, "tps": 82197.65738691934}

After (~108493 TPS) => ~24.9% boost in build TPS 🎉

[1] block generation {"t": "4.482730875s", "avg(ms)": 91, "tps": 111539.1519014623}
[2] block generation {"t": "4.759061083s", "avg(ms)": 95, "tps": 105062.74058680747}
[3] block generation {"t": "4.592278459s", "avg(ms)": 93, "tps": 108878.41503167001}
[4] block generation {"t": "4.618432834s", "avg(ms)": 94, "tps": 108261.83209141805}
[5] block generation {"t": "4.598757792s", "avg(ms)": 93, "tps": 108725.01284364228}

TODO

  • Build -> Top
  • Remove account purging (avoid iteration and just wait for cleanup naturally)

@patrick-ogrady
Copy link
Contributor Author

patrick-ogrady commented Aug 12, 2023

(before interleaving fetch with execute but with leases)

image

@patrick-ogrady
Copy link
Contributor Author

patrick-ogrady commented Aug 12, 2023

Values here aren't directly comparable because load is slightly different between runs

Before

[08-12|11:26:47.069] INFO load/load_test.go:325 block generation {"t": "6.026229334s", "avg(ms)": 122, "tps": 82970.62263777432}

After

[08-12|11:28:58.212] INFO load/load_test.go:325 block generation {"t": "4.572423958s", "avg(ms)": 93, "tps": 109351.18978308879}```

@patrick-ogrady patrick-ogrady changed the title [mempool] Use Lease instead of Build [mempool] Use Lease instead of Build + Prefetch Mempool Txs/State + Batch IsRepeat Aug 12, 2023
@patrick-ogrady patrick-ogrady self-assigned this Aug 12, 2023
@patrick-ogrady patrick-ogrady changed the title [mempool] Use Lease instead of Build + Prefetch Mempool Txs/State + Batch IsRepeat [mempool] Use Stream instead of Build + Prefetch Mempool Txs/State + Batch IsRepeat Aug 13, 2023
@patrick-ogrady patrick-ogrady marked this pull request as ready for review August 13, 2023 18:17
@patrick-ogrady patrick-ogrady merged commit d9cc0a7 into main Aug 13, 2023
@patrick-ogrady patrick-ogrady deleted the mempool-leases branch August 13, 2023 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant